www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - DMD performance dashboard is live now

reply Abul Hossain Khan <abulkhan19175 gmail.com> writes:
Hi everyone,

Quick update on the performance regression publisher project from 
GSoC. The dashboard is now up and running at 
https://perf.dlang.org

Every merge to dmd master gets measured by the perf workflow, the 
result goes into a data repo 
https://github.com/dlang/dmd-perf-data and the web page is 
showing those data.

Two improvements also went into the PR bot after GSoC:

- https://github.com/dlang/dmd/pull/23834 - The PR check was 
reusing base's PGO profile for the head build, so some PRs got 
big deltas that weren't real (one showed +7.49% on hello.d, after 
merge the master run said 0.00%). Now both sides train their own 
profile, so the numbers are a lot more accurate.
- https://github.com/dlang/dmd/pull/23884 - base and head are now 
built in parallel, nearly halves build time.

I will keep an eye on the dashboard and look into any spikes, 
that will be be interesting.
Thanks to Dennis for mentoring the whole thing, and Mike for 
setting things up, and to the whole D community. And as always, 
feel free to leave any feedback or suggestions!
Sep 23
next sibling parent reply user1234 <user1234 12.de> writes:
On Wednesday, 23 September 2026 at 15:01:52 UTC, Abul Hossain 
Khan wrote:

 Hi everyone,

 Quick update on the performance regression publisher project 
 from GSoC. The dashboard is now up and running at 
 https://perf.dlang.org

 [...]

 Thanks to Dennis for mentoring the whole thing, and Mike for 
 setting things up, and to the whole D community. And as always, 
 feel free to leave any feedback or suggestions!


Congratz, nice work. Is there any plan to eventually block a PR 
if the delta is too suspiciously positive ?
Sep 23
parent Abul Hossain Khan <abulkhan19175 gmail.com> writes:
 Congratz, nice work. Is there any plan to eventually block a PR 
 if the delta is too suspiciously positive ?


Thank you so much, I don't think blocking makes sense, because a 
new feature implementation can cost some performance


 that someone could look at it with fresh eyes and not need to 
 go to the Info page


Good point, I'll add axis labels and more clearer titles


 It might also be interesting to integrate something like this 
 into it (just for D)


Looks interesting, I'll have a look at it.


 I see some graphs are a bit noisy.


Yeah, those are wall time and peak RSS. The main metrics are 
instruction counts from cachegrind, which barely move between 
runs (around 0.001%). Wall time is noisy on shared runners 
however many times you run it, so it's more informational.


 I would suggest moving the benchmarks to a dedicated server


Yes, it runs on GitHub Actions, and that's the reason the main 
metric is instruction count and not time. A dedicated server 
would be really great for this, but that would be up to the 
Organisation DLF.


 are you pinning the Phobos version?


No we are not, I had suggested pinning it before, but Dennis 
pointed out that an old pinned Phobos has a large maintenance 
cost, since dmd and Phobos kinda change together. What we can do 
is store the Phobos commit in each record, so if a chart jumps 
because of Phobos, everyone can see it.

Thank you so much everyone for the suggestion, Really Appreciated!
Sep 28
prev sibling next sibling parent M.M. <matus email.cz> writes:
On Wednesday, 23 September 2026 at 15:01:52 UTC, Abul Hossain 
Khan wrote:

 Hi everyone,

 Quick update on the performance regression publisher project 
 from GSoC. The dashboard is now up and running at 
 https://perf.dlang.org

 [...]


Nice work!
Sep 24
prev sibling next sibling parent reply jmh530 <john.michael.hall gmail.com> writes:
On Wednesday, 23 September 2026 at 15:01:52 UTC, Abul Hossain 
Khan wrote:

 Hi everyone,

 Quick update on the performance regression publisher project 
 from GSoC. The dashboard is now up and running at 
 https://perf.dlang.org


Looks good. In terms of presentation, you might consider adding y 
axis labels and giving the titles more descriptive names (and 
centering them). The more that someone could look at it with 
fresh eyes and not need to go to the Info page, probably the 
better.

It might also be interesting to integrate something like this 
into it (just for D)
https://github.com/nordlow/compiler-benchmark
such that you can track runtime performance over time.
Sep 24
parent Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Thursday, 24 September 2026 at 13:49:07 UTC, jmh530 wrote:

 It might also be interesting to integrate something like this 
 into it (just for D)
 https://github.com/nordlow/compiler-benchmark
 such that you can track runtime performance over time.


Incidentally, I've been spending hours improving 
compiler-benchmark this week. I've added more languages and now 
working on parallell execution across multiple processes.
Sep 24
prev sibling next sibling parent Andrea Fontana <nospam example.org> writes:
On Wednesday, 23 September 2026 at 15:01:52 UTC, Abul Hossain 
Khan wrote:

 Hi everyone,

 Quick update on the performance regression publisher project 
 from GSoC. The dashboard is now up and running at 
 https://perf.dlang.org



Nice! I see some graphs are a bit noisy. Did you run the tests 
just once? Maybe running them multiple times and taking the 
median would do the trick.

Andrea
Sep 25
prev sibling parent reply libxmoc <libxmoc gmail.com> writes:
That's awesome, thanks for your work!

I was wondering tho, what are you using to run the benchmarks?

If you are using GitHub Actions, I would suggest moving the 
benchmarks to a dedicated server.

GitHub Actions runners run on shared/ephemeral virtual machines, 
so their performance will vary between runs.

That make them less reliable for detecting small performance 
changes or regressions.

A dedicated machine would give you a much more controlled and 
reproducibel environment.
Sep 26
parent libxmoc <libxmoc gmail.com> writes:
On Saturday, 26 September 2026 at 07:03:52 UTC, libxmoc wrote:

 That's awesome, thanks for your work!

 I was wondering tho, what are you using to run the benchmarks?

 If you are using GitHub Actions, I would suggest moving the 
 benchmarks to a dedicated server.

 GitHub Actions runners run on shared/ephemeral virtual 
 machines, so their performance will vary between runs.

 That make them less reliable for detecting small performance 
 changes or regressions.

 A dedicated machine would give you a much more controlled and 
 reproducibel environment.


Also, are you pinning the Phobos version? Otherwise, a change in 
Phobos between two runs could affect the results too.
Sep 26