www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Profiling DMD's Compilation Time with dmdprof

reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
This is a tool + article I wrote in February, but never got 
around to finishing / publishing until today.

https://blog.thecybershadow.net/2018/02/07/dmdprof/

Hopefully someone will find it useful.
Nov 06 2018
next sibling parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Nov 06, 2018 at 06:00:22PM +0000, Vladimir Panteleev via
Digitalmars-d-announce wrote:

 This is a tool + article I wrote in February, but never got around to
 finishing / publishing until today.
 
 https://blog.thecybershadow.net/2018/02/07/dmdprof/
 
 Hopefully someone will find it useful.


I don't have the time to look into this right now, but at a cursory
glance, WOW.  This is awesome!  It looks like it would be really useful
one day when I try to tackle the dmd-on-lowmem-system problem again.
This will greatly help identify which Phobos modules cause big
slowdowns on a low-memory host system.


T

-- 
Recently, our IT department hired a bug-fix engineer. He used to work for
Volkswagen.
Nov 06 2018
next sibling parent Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Tuesday, 6 November 2018 at 19:01:58 UTC, H. S. Teoh wrote:

 I don't have the time to look into this right now, but at a 
 cursory glance, WOW.  This is awesome!  It looks like it would 
 be really useful one day when I try to tackle the 
 dmd-on-lowmem-system problem again. This will greatly help 
 identify which Phobos modules cause big slowdowns on a 
 low-memory host system.


Great. Also, assuming that DMD still never deallocates, it should 
be pretty easy to turn this into a memory usage profiler by 
measuring the delta between the resident set size in samples.
Nov 06 2018
prev sibling parent welkam <wwwelkam gmail.com> writes:
On Tuesday, 6 November 2018 at 19:01:58 UTC, H. S. Teoh wrote:

 It looks like it would be really useful one day when I try to 
 tackle the dmd-on-lowmem-system problem again.


Based on my profiling it seems that most memory is allocated in 
void importAll(Scope* sc) found in attrib.d . A person with more 
knowledge of DMD source could create new allocator for scope data 
and when its no longer needed just dealloc all. My intuition says 
that after IR is generated we no longer need scope information.

Here is profile data for simple file.
https://imgur.com/a/ROa6JNd
Nov 08 2018
prev sibling next sibling parent reply Atila Neves <atila.neves gmail.com> writes:
On Tuesday, 6 November 2018 at 18:00:22 UTC, Vladimir Panteleev 
wrote:

 This is a tool + article I wrote in February, but never got 
 around to finishing / publishing until today.

 https://blog.thecybershadow.net/2018/02/07/dmdprof/

 Hopefully someone will find it useful.


Awesome, great work!

I really really hate waiting for the compiler.
Nov 06 2018
parent reply "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Tue, Nov 06, 2018 at 07:44:41PM +0000, Atila Neves via
Digitalmars-d-announce wrote:

 On Tuesday, 6 November 2018 at 18:00:22 UTC, Vladimir Panteleev wrote:

 This is a tool + article I wrote in February, but never got around
 to finishing / publishing until today.
 
 https://blog.thecybershadow.net/2018/02/07/dmdprof/
 
 Hopefully someone will find it useful.


 
 Awesome, great work!
 
 I really really hate waiting for the compiler.


OTOH, I really really hate that the compiler, in the name of faster
compilation, eats up all available RAM and gets OOM-killed on a low
memory system, so no amount of waiting will get me an executable.


T

-- 
Famous last words: I wonder what will happen if I do *this*...
Nov 06 2018
parent reply Patrick Schluter <Patrick.Schluter bbox.fr> writes:
On Tuesday, 6 November 2018 at 21:52:30 UTC, H. S. Teoh wrote:

 On Tue, Nov 06, 2018 at 07:44:41PM +0000, Atila Neves via 
 Digitalmars-d-announce wrote:

 On Tuesday, 6 November 2018 at 18:00:22 UTC, Vladimir 
 Panteleev wrote:

 This is a tool + article I wrote in February, but never got 
 around to finishing / publishing until today.
 
 https://blog.thecybershadow.net/2018/02/07/dmdprof/
 
 Hopefully someone will find it useful.


 
 Awesome, great work!
 
 I really really hate waiting for the compiler.


 OTOH, I really really hate that the compiler, in the name of 
 faster compilation, eats up all available RAM and gets 
 OOM-killed on a low memory system, so no amount of waiting will 
 get me an executable.


Now that the compiler is completely in D, wouldn't it be a good 
idea to activate the GC in the compiler. I know that it requires 
some care for bootstrapping the compiler when there are 
dependencies to the D runtime, but the compiler would be an 
excellent example of the advantage of the GC (i.e. dumb fast 
allocations as long as there's memory, collection when no memory 
left which is miles away better than to get OOM-killed).
Nov 07 2018
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 08/11/2018 1:46 AM, Patrick Schluter wrote:

 Now that the compiler is completely in D, wouldn't it be a good idea to 
 activate the GC in the compiler. I know that it requires some care for 
 bootstrapping the compiler when there are dependencies to the D runtime, 
 but the compiler would be an excellent example of the advantage of the 
 GC (i.e. dumb fast allocations as long as there's memory, collection 
 when no memory left which is miles away better than to get OOM-killed).


No, that would be a bad idea currently.

https://issues.dlang.org/show_bug.cgi?id=18811
Nov 07 2018
parent reply Neia Neutuladh <neia ikeran.org> writes:
On Thu, 08 Nov 2018 01:49:49 +1300, rikki cattermole wrote:

 On 08/11/2018 1:46 AM, Patrick Schluter wrote:

 Now that the compiler is completely in D, wouldn't it be a good idea to
 activate the GC in the compiler. I know that it requires some care for
 bootstrapping the compiler when there are dependencies to the D
 runtime,
 but the compiler would be an excellent example of the advantage of the
 GC (i.e. dumb fast allocations as long as there's memory, collection
 when no memory left which is miles away better than to get OOM-killed).


 
 No, that would be a bad idea currently.
 
 https://issues.dlang.org/show_bug.cgi?id=18811


That issue describes a memory leak. A memory leak of half a megabyte per 
full compiler invocation on a small file should still result in a lot of 
code being compilable on low-end machines that currently isn't.
Nov 07 2018
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 08/11/2018 4:17 AM, Neia Neutuladh wrote:

 On Thu, 08 Nov 2018 01:49:49 +1300, rikki cattermole wrote:

 On 08/11/2018 1:46 AM, Patrick Schluter wrote:

 Now that the compiler is completely in D, wouldn't it be a good idea to
 activate the GC in the compiler. I know that it requires some care for
 bootstrapping the compiler when there are dependencies to the D
 runtime,
 but the compiler would be an excellent example of the advantage of the
 GC (i.e. dumb fast allocations as long as there's memory, collection
 when no memory left which is miles away better than to get OOM-killed).


 No, that would be a bad idea currently.

 https://issues.dlang.org/show_bug.cgi?id=18811


 
 That issue describes a memory leak. A memory leak of half a megabyte per
 full compiler invocation on a small file should still result in a lot of
 code being compilable on low-end machines that currently isn't.


Its a symptom of a larger set of problems. The frontend is not quite 
ready to have the GC turned on full time.

Based upon my testing, that little memory leak prevents pretty much 
*all* memory allocated by the GC to not be collected. I don't know why, 
but for some reason it pins it. Mind you, my testing could have been 
flawed, and needs more eyes on it *shrug*.
Nov 07 2018
parent Neia Neutuladh <neia ikeran.org> writes:
On Thu, 08 Nov 2018 14:35:29 +1300, rikki cattermole wrote:

 Its a symptom of a larger set of problems. The frontend is not quite
 ready to have the GC turned on full time.
 
 Based upon my testing, that little memory leak prevents pretty much
 *all* memory allocated by the GC to not be collected. I don't know why,
 but for some reason it pins it. Mind you, my testing could have been
 flawed, and needs more eyes on it *shrug*.


In the short term, that means turning on the GC won't do much, but it at 
least probably won't hurt.

In the long term, that bug might be covering up crashes and memory 
corruption, so turning it on by default is not a great idea.
Nov 07 2018
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On Wed, Nov 7, 2018 at 8:16 PM Manu <turkeyman gmail.com> wrote:

 On Tue, Nov 6, 2018 at 10:05 AM Vladimir Panteleev via
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 This is a tool + article I wrote in February, but never got
 around to finishing / publishing until today.

 https://blog.thecybershadow.net/2018/02/07/dmdprof/

 Hopefully someone will find it useful.


 "Indeed, a clean build of DMD itself (about 170’000 lines of D and
 120’000 lines of C/C++) takes no longer than 4 seconds to build on a
 rather average developer machine."

 ...what!? DMD takes me... (compiling) ... 1 minute 40 seconds to build!
 And because DMD does all-files-at-once compilation, rather than
 separate compilation for each source file, whenever you change just
 one line in one file, you incur that entire build time, every time,
 because it can't just rebuild the one source file that changed. You
 also can't do multi-processor builds with all-in-one build strategies.

 4 seconds? That's just untrue. D is actually kinda slow these days...
 In my experience it's slower than modern C++ compilers by quite a lot.


Also, in my experience, DMD seems to build a LOT slower now that it's
in D than it used to when it was C++.
Nov 07 2018
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On Wed, Nov 7, 2018 at 8:18 PM Manu <turkeyman gmail.com> wrote:

 On Wed, Nov 7, 2018 at 8:16 PM Manu <turkeyman gmail.com> wrote:

 On Tue, Nov 6, 2018 at 10:05 AM Vladimir Panteleev via
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 This is a tool + article I wrote in February, but never got
 around to finishing / publishing until today.

 https://blog.thecybershadow.net/2018/02/07/dmdprof/

 Hopefully someone will find it useful.


 "Indeed, a clean build of DMD itself (about 170’000 lines of D and
 120’000 lines of C/C++) takes no longer than 4 seconds to build on a
 rather average developer machine."

 ...what!? DMD takes me... (compiling) ... 1 minute 40 seconds to build!
 And because DMD does all-files-at-once compilation, rather than
 separate compilation for each source file, whenever you change just
 one line in one file, you incur that entire build time, every time,
 because it can't just rebuild the one source file that changed. You
 also can't do multi-processor builds with all-in-one build strategies.

 4 seconds? That's just untrue. D is actually kinda slow these days...
 In my experience it's slower than modern C++ compilers by quite a lot.


 Also, in my experience, DMD seems to build a LOT slower now that it's
 in D than it used to when it was C++.


Oh, and also, nice work Vladimir! This is awesome! :)
Nov 07 2018
prev sibling next sibling parent reply Manu <turkeyman gmail.com> writes:
On Tue, Nov 6, 2018 at 10:05 AM Vladimir Panteleev via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 This is a tool + article I wrote in February, but never got
 around to finishing / publishing until today.

 https://blog.thecybershadow.net/2018/02/07/dmdprof/

 Hopefully someone will find it useful.


"Indeed, a clean build of DMD itself (about 170’000 lines of D and
120’000 lines of C/C++) takes no longer than 4 seconds to build on a
rather average developer machine."

...what!? DMD takes me... (compiling) ... 1 minute 40 seconds to build!
And because DMD does all-files-at-once compilation, rather than
separate compilation for each source file, whenever you change just
one line in one file, you incur that entire build time, every time,
because it can't just rebuild the one source file that changed. You
also can't do multi-processor builds with all-in-one build strategies.

4 seconds? That's just untrue. D is actually kinda slow these days...
In my experience it's slower than modern C++ compilers by quite a lot.
Nov 07 2018
next sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Thursday, 8 November 2018 at 04:16:44 UTC, Manu wrote:

 ...what!? DMD takes me... (compiling) ... 1 minute 40 seconds 
 to build! And because DMD does all-files-at-once compilation, 
 rather than separate compilation for each source file, whenever 
 you change just one line in one file, you incur that entire 
 build time, every time, because it can't just rebuild the one 
 source file that changed. You also can't do multi-processor 
 builds with all-in-one build strategies.

 4 seconds? That's just untrue. D is actually kinda slow these 
 days... In my experience it's slower than modern C++ compilers 
 by quite a lot.


It was definitely about 4 seconds not too long ago, a few years 
at most.

It does seem to take more time now; I wonder why.
Nov 07 2018
next sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Thursday, 8 November 2018 at 06:08:20 UTC, Vladimir Panteleev 
wrote:

 It was definitely about 4 seconds not too long ago, a few years 
 at most.


No, it's still 4 seconds.

digger --offline --config-file=/dev/null -j auto -c 
local.cache=none build     7.31s user 1.51s system 203% cpu 4.340 
total


 It does seem to take more time now; I wonder why.


If it takes longer, then it's probably because it's being built 
in one CPU core, or in the release build.
Nov 07 2018
parent reply Manu <turkeyman gmail.com> writes:
On Wed, Nov 7, 2018 at 10:30 PM Vladimir Panteleev via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On Thursday, 8 November 2018 at 06:08:20 UTC, Vladimir Panteleev
 wrote:

 It was definitely about 4 seconds not too long ago, a few years
 at most.


 No, it's still 4 seconds.

 digger --offline --config-file=/dev/null -j auto -c
 local.cache=none build     7.31s user 1.51s system 203% cpu 4.340
 total


 It does seem to take more time now; I wonder why.


 If it takes longer, then it's probably because it's being built
 in one CPU core, or in the release build.


https://youtu.be/msWuRlD3zy0

DMD only builds with one core, since it builds altogether.
And all builds are release builds... what good is a debug build? DMD
is unbelievably slow in debug. If it wasn't already slow enough... if
I try and build with a debug build, it takes closer to 5 minutes.

I suspect one part of the problem is that DMD used to be built with a
C compiler, and now it's built with DMD... it really should be built
with LDC at least?
Nov 07 2018
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 8 November 2018 at 07:54:56 UTC, Manu wrote:

 On Wed, Nov 7, 2018 at 10:30 PM Vladimir Panteleev via 
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> 
 wrote:

 On Thursday, 8 November 2018 at 06:08:20 UTC, Vladimir 
 Panteleev wrote:

 It was definitely about 4 seconds not too long ago, a few 
 years at most.


 No, it's still 4 seconds.

 digger --offline --config-file=/dev/null -j auto -c 
 local.cache=none build     7.31s user 1.51s system 203% cpu 
 4.340 total


 It does seem to take more time now; I wonder why.


 If it takes longer, then it's probably because it's being 
 built in one CPU core, or in the release build.


 https://youtu.be/msWuRlD3zy0


Lol, I saw that link and figured it was either some comedy video, 
like the Python ones Walter sometimes posts, or you were actually 
showing us how long it takes. Pretty funny to see the latter.


 DMD only builds with one core, since it builds altogether.


Yes, but your build time is unusually long even with one core. 
Are the D backend and frontend at least built in parallel to each 
other? It doesn't seem to be even doing that, though they're 
separate invocations of DMD.


 And all builds are release builds... what good is a debug 
 build? DMD
 is unbelievably slow in debug. If it wasn't already slow 
 enough... if
 I try and build with a debug build, it takes closer to 5 
 minutes.

 I suspect one part of the problem is that DMD used to be built 
 with a C compiler, and now it's built with DMD... it really 
 should be built with LDC at least?


Could be part of the problem on Windows, dunno.
Nov 08 2018
parent reply Manu <turkeyman gmail.com> writes:
On Thu, Nov 8, 2018 at 12:10 AM Joakim via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 On Thursday, 8 November 2018 at 07:54:56 UTC, Manu wrote:

 On Wed, Nov 7, 2018 at 10:30 PM Vladimir Panteleev via
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com>
 wrote:

 On Thursday, 8 November 2018 at 06:08:20 UTC, Vladimir
 Panteleev wrote:

 It was definitely about 4 seconds not too long ago, a few
 years at most.


 No, it's still 4 seconds.

 digger --offline --config-file=/dev/null -j auto -c
 local.cache=none build     7.31s user 1.51s system 203% cpu
 4.340 total


 It does seem to take more time now; I wonder why.


 If it takes longer, then it's probably because it's being
 built in one CPU core, or in the release build.


 https://youtu.be/msWuRlD3zy0


 Lol, I saw that link and figured it was either some comedy video,
 like the Python ones Walter sometimes posts, or you were actually
 showing us how long it takes. Pretty funny to see the latter.


It's not so funny when every one-line tweak burns 2 minutes of my life away.


 DMD only builds with one core, since it builds altogether.


 Yes, but your build time is unusually long even with one core.
 Are the D backend and frontend at least built in parallel to each
 other?


That doesn't matter, you can clearly see the backend built in less
than 2 seconds.


 It doesn't seem to be even doing that, though they're
 separate invocations of DMD.


I didn't configure the build infrastructure!


 And all builds are release builds... what good is a debug
 build? DMD
 is unbelievably slow in debug. If it wasn't already slow
 enough... if
 I try and build with a debug build, it takes closer to 5
 minutes.

 I suspect one part of the problem is that DMD used to be built
 with a C compiler, and now it's built with DMD... it really
 should be built with LDC at least?


 Could be part of the problem on Windows, dunno.


Well... ffs... people need to care about this! >_<
Nov 08 2018
parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 8 November 2018 at 08:29:28 UTC, Manu wrote:

 On Thu, Nov 8, 2018 at 12:10 AM Joakim via 
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> 
 wrote:

 On Thursday, 8 November 2018 at 07:54:56 UTC, Manu wrote:

 On Wed, Nov 7, 2018 at 10:30 PM Vladimir Panteleev via 
 Digitalmars-d-announce 
 <digitalmars-d-announce puremagic.com> wrote:

 On Thursday, 8 November 2018 at 06:08:20 UTC, Vladimir 
 Panteleev wrote:

 It was definitely about 4 seconds not too long ago, a few 
 years at most.


 No, it's still 4 seconds.

 digger --offline --config-file=/dev/null -j auto -c 
 local.cache=none build     7.31s user 1.51s system 203% cpu 
 4.340 total


 It does seem to take more time now; I wonder why.


 If it takes longer, then it's probably because it's being 
 built in one CPU core, or in the release build.


 https://youtu.be/msWuRlD3zy0


 Lol, I saw that link and figured it was either some comedy 
 video, like the Python ones Walter sometimes posts, or you 
 were actually showing us how long it takes. Pretty funny to 
 see the latter.


 It's not so funny when every one-line tweak burns 2 minutes of 
 my life away.


I was laughing that you actually proved your point with direct 
video evidence, obviously it's sad that it takes so long.


 DMD only builds with one core, since it builds altogether.


 Yes, but your build time is unusually long even with one core. 
 Are the D backend and frontend at least built in parallel to 
 each other?


 That doesn't matter, you can clearly see the backend built in 
 less than 2 seconds.


The C/C++ files in the beginning are built very fast, but the D 
files in the backend appear to take much longer, kicking in at 
1:18 of your video and then the next compilation step starts at 
1:40.

I suspect part of the problem is that your build is being done 
completely serially, even for separate compilation. I have no 
experience with VS, so I don't know why that is.


 It doesn't seem to be even doing that, though they're separate 
 invocations of DMD.


 I didn't configure the build infrastructure!


Maybe you can? I have no experience with VS, but surely it has 
some equivalent of ninja -j5?


 And all builds are release builds... what good is a debug
 build? DMD
 is unbelievably slow in debug. If it wasn't already slow
 enough... if
 I try and build with a debug build, it takes closer to 5
 minutes.

 I suspect one part of the problem is that DMD used to be 
 built with a C compiler, and now it's built with DMD... it 
 really should be built with LDC at least?


 Could be part of the problem on Windows, dunno.


 Well... ffs... people need to care about this! >_<


I agree that the official release of DMD for Windows should be 
faster, and we should be building it with ldc... if that's the 
problem.
Nov 08 2018
parent Manu <turkeyman gmail.com> writes:
On Thu, Nov 8, 2018 at 12:55 AM Joakim via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 On Thursday, 8 November 2018 at 08:29:28 UTC, Manu wrote:

 On Thu, Nov 8, 2018 at 12:10 AM Joakim via
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com>
 wrote:

 On Thursday, 8 November 2018 at 07:54:56 UTC, Manu wrote:


 I didn't configure the build infrastructure!


 Maybe you can? I have no experience with VS, but surely it has
 some equivalent of ninja -j5?


msbuild does parallel builds quite effectively. I expect it perceives
a dependency between jobs which cause it to serialise. Maybe there's a
legit dependency, or maybe the msbuild script has a problem? Either
way, it's not acceptable.
I would log this is maximum priority bug.
(https://issues.dlang.org/show_bug.cgi?id=19377)


 And all builds are release builds... what good is a debug
 build? DMD
 is unbelievably slow in debug. If it wasn't already slow
 enough... if
 I try and build with a debug build, it takes closer to 5
 minutes.

 I suspect one part of the problem is that DMD used to be
 built with a C compiler, and now it's built with DMD... it
 really should be built with LDC at least?


 Could be part of the problem on Windows, dunno.


 Well... ffs... people need to care about this! >_<


 I agree that the official release of DMD for Windows should be
 faster, and we should be building it with ldc... if that's the
 problem.


I think it's a combination of problems, and primary problem being
criminal negligence!
Nov 08 2018
prev sibling next sibling parent reply Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Thursday, 8 November 2018 at 07:54:56 UTC, Manu wrote:

 On Wed, Nov 7, 2018 at 10:30 PM Vladimir Panteleev via 
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> 
 wrote:

 On Thursday, 8 November 2018 at 06:08:20 UTC, Vladimir 
 Panteleev wrote:

 It was definitely about 4 seconds not too long ago, a few 
 years at most.


 No, it's still 4 seconds.

 digger --offline --config-file=/dev/null -j auto -c 
 local.cache=none build     7.31s user 1.51s system 203% cpu 
 4.340 total


 It does seem to take more time now; I wonder why.


 If it takes longer, then it's probably because it's being 
 built in one CPU core, or in the release build.


 https://youtu.be/msWuRlD3zy0

 DMD only builds with one core, since it builds altogether.
 And all builds are release builds... what good is a debug 
 build? DMD
 is unbelievably slow in debug. If it wasn't already slow 
 enough... if
 I try and build with a debug build, it takes closer to 5 
 minutes.

 I suspect one part of the problem is that DMD used to be built 
 with a C compiler, and now it's built with DMD... it really 
 should be built with LDC at least?


The problem definitely looks like is with your Windows. Perhaps 
it's caused by an antivirus software? For the record on my 
machine it's consistently below 4sec. See:
https://asciinema.org/a/G5lLRPsLLMb5IaqPrmhZagop4
Nov 08 2018
parent reply John Chapman <johnch_atms hotmail.com> writes:
On Thursday, 8 November 2018 at 14:33:34 UTC, Petar Kirov 
[ZombineDev] wrote:

 On Thursday, 8 November 2018 at 07:54:56 UTC, Manu wrote:

 On Wed, Nov 7, 2018 at 10:30 PM Vladimir Panteleev via 
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> 
 wrote:

 On Thursday, 8 November 2018 at 06:08:20 UTC, Vladimir 
 Panteleev wrote:

 It was definitely about 4 seconds not too long ago, a few 
 years at most.


 No, it's still 4 seconds.

 digger --offline --config-file=/dev/null -j auto -c 
 local.cache=none build     7.31s user 1.51s system 203% cpu 
 4.340 total


 It does seem to take more time now; I wonder why.


 If it takes longer, then it's probably because it's being 
 built in one CPU core, or in the release build.


 https://youtu.be/msWuRlD3zy0

 DMD only builds with one core, since it builds altogether.
 And all builds are release builds... what good is a debug 
 build? DMD
 is unbelievably slow in debug. If it wasn't already slow 
 enough... if
 I try and build with a debug build, it takes closer to 5 
 minutes.

 I suspect one part of the problem is that DMD used to be built 
 with a C compiler, and now it's built with DMD... it really 
 should be built with LDC at least?


 The problem definitely looks like is with your Windows. Perhaps 
 it's caused by an antivirus software? For the record on my 
 machine it's consistently below 4sec. See:
 https://asciinema.org/a/G5lLRPsLLMb5IaqPrmhZagop4


I don't think it's just *his* Windows - DMD builds on my Windows 
10 Core i3 laptop with anti-virus disabled in about 1m 40s too. 
And it's always taken about this long, even when the source was 
C++. I'm tempted to try it on Linux on the same machine to see 
what I'm missing out on.
Nov 08 2018
parent Petar Kirov [ZombineDev] <petar.p.kirov gmail.com> writes:
On Thursday, 8 November 2018 at 16:51:55 UTC, John Chapman wrote:

 I don't think it's just *his* Windows - DMD builds on my 
 Windows 10 Core i3 laptop with anti-virus disabled in about 1m 
 40s too. And it's always taken about this long, even when the 
 source was C++. I'm tempted to try it on Linux on the same 
 machine to see what I'm missing out on.


BTW, last time I tried WSL it had similar performance to Linux, 
IIRC. Perhaps this is why I haven't noticed such slow downs when 
building dmd on the machines I tried.
Nov 08 2018
prev sibling next sibling parent Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Thursday, 8 November 2018 at 07:54:56 UTC, Manu wrote:

 https://youtu.be/msWuRlD3zy0

 DMD only builds with one core, since it builds altogether.
 And all builds are release builds... what good is a debug 
 build? DMD
 is unbelievably slow in debug. If it wasn't already slow 
 enough... if
 I try and build with a debug build, it takes closer to 5 
 minutes.


I think the default posix.mak target is neither a debug nor a 
release build, meaning it is neither optimized nor has the 
additional asserts/invariants of a debug build. Not sure the 
Visual Studio solution has that, but perhaps it could be an 
acceptable middle ground.


 I suspect one part of the problem is that DMD used to be built 
 with a C compiler, and now it's built with DMD...


Well, my latest test was against master, that even has parts of 
the backend apparently converted to D, so I'm not sure that's 
relevant.
Nov 08 2018
prev sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Thursday, 8 November 2018 at 07:54:56 UTC, Manu wrote:

 And all builds are release builds... what good is a debug 
 build? DMD
 is unbelievably slow in debug. If it wasn't already slow 
 enough... if
 I try and build with a debug build, it takes closer to 5 
 minutes.


I just got to try a side-by-side comparison of a release and 
debug (as in `make -f posix.mak BUILD=debug`) build of DMD.

With a 25KLOC project, the debug build is only 36% slower.

Maybe the experience on Windows / Visual Studio is very different.
Nov 14 2018
parent reply Manu <turkeyman gmail.com> writes:
Wed, Nov 14, 2018 at 12:25 AM Vladimir Panteleev via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On Thursday, 8 November 2018 at 07:54:56 UTC, Manu wrote:

 And all builds are release builds... what good is a debug
 build? DMD
 is unbelievably slow in debug. If it wasn't already slow
 enough... if
 I try and build with a debug build, it takes closer to 5
 minutes.


 I just got to try a side-by-side comparison of a release and
 debug (as in `make -f posix.mak BUILD=debug`) build of DMD.

 With a 25KLOC project, the debug build is only 36% slower.

 Maybe the experience on Windows / Visual Studio is very different.


I'm not sure how VisualStudio (read: MSBuild) should behave
differently than make?
It's not like the build script is taking a long time, it's the
invocation of DMD that takes 100% of that time.

36% slower seems highly optimistic? Perhaps you're building a debug
build of DMD with a debug build of DMD? I guess that wouldn't be so
slow... I suspect it's the optimiser that's very slow?
Nov 15 2018
next sibling parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Thursday, 15 November 2018 at 19:18:27 UTC, Manu wrote:

 I'm not sure how VisualStudio (read: MSBuild) should behave
 differently than make?
 It's not like the build script is taking a long time, it's the
 invocation of DMD that takes 100% of that time.


That seems to take about half the time for me (2 out of the 4 
seconds).


 36% slower seems highly optimistic? Perhaps you're building a 
 debug build of DMD with a debug build of DMD? I guess that 
 wouldn't be so slow... I suspect it's the optimiser that's very 
 slow?


36% slower for compiling a real-life program with the built D 
compiler. The comparison is between a normal and debug build of 
DMD (used to build the program).
Nov 15 2018
parent reply Manu <turkeyman gmail.com> writes:
On Thu, Nov 15, 2018 at 8:00 PM Vladimir Panteleev via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On Thursday, 15 November 2018 at 19:18:27 UTC, Manu wrote:

 I'm not sure how VisualStudio (read: MSBuild) should behave
 differently than make?
 It's not like the build script is taking a long time, it's the
 invocation of DMD that takes 100% of that time.


 That seems to take about half the time for me (2 out of the 4
 seconds).


 36% slower seems highly optimistic? Perhaps you're building a
 debug build of DMD with a debug build of DMD? I guess that
 wouldn't be so slow... I suspect it's the optimiser that's very
 slow?


 36% slower for compiling a real-life program with the built D
 compiler. The comparison is between a normal and debug build of
 DMD (used to build the program).


What was the batch size for module grouping? I wonder if batching too
many modules together into a single compiler invocation makes too much
data for the optimiser? Is optimisation time strictly linear with the
size of the AST? Or is having more functions available for the inliner
to consider increasingly expensive?
Nov 16 2018
parent Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Saturday, 17 November 2018 at 06:21:20 UTC, Manu wrote:

 What was the batch size for module grouping?


I just use dmd -i, which compiles everything at once (except 
Phobos/Druntime).
Nov 17 2018
prev sibling parent reply welkam <wwwelkam gmail.com> writes:
I just updated DMD on my windows and tried to compile hello 
world. It took 7.4 sec. Something is not right. Can anyone 
reproduce? On the same mashine running linux it compiles and runs 
in 0.1 sec

PS C:\Users\Welkam\Desktop\Projects> Measure-Command {dmd -run 
main.d}


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 7
Milliseconds      : 418
Ticks             : 74184371
TotalDays         : 8.58615405092593E-05
TotalHours        : 0.00206067697222222
TotalMinutes      : 0.123640618333333
TotalSeconds      : 7.4184371
TotalMilliseconds : 7418.4371
Nov 17 2018
parent reply rikki cattermole <rikki cattermole.co.nz> writes:
On 18/11/2018 1:29 AM, welkam wrote:

 I just updated DMD on my windows and tried to compile hello world. It 
 took 7.4 sec. Something is not right. Can anyone reproduce? On the same 
 mashine running linux it compiles and runs in 0.1 sec
 
 PS C:\Users\Welkam\Desktop\Projects> Measure-Command {dmd -run main.d}
 
 
 Days              : 0
 Hours             : 0
 Minutes           : 0
 Seconds           : 7
 Milliseconds      : 418
 Ticks             : 74184371
 TotalDays         : 8.58615405092593E-05
 TotalHours        : 0.00206067697222222
 TotalMinutes      : 0.123640618333333
 TotalSeconds      : 7.4184371
 TotalMilliseconds : 7418.4371


Just in case, did you disable your anti-virus first?
Nov 17 2018
parent welkam <wwwelkam gmail.com> writes:
On Saturday, 17 November 2018 at 12:36:30 UTC, rikki cattermole 
wrote:

 Just in case, did you disable your anti-virus first?


Had to edit registry to turn that thing off and it still some 
times popup. Microsoft is so annoying.
https://imgur.com/a/x3cKjZm
Nov 17 2018
prev sibling next sibling parent Manu <turkeyman gmail.com> writes:
On Wed, Nov 7, 2018 at 10:10 PM Vladimir Panteleev via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On Thursday, 8 November 2018 at 04:16:44 UTC, Manu wrote:

 ...what!? DMD takes me... (compiling) ... 1 minute 40 seconds
 to build! And because DMD does all-files-at-once compilation,
 rather than separate compilation for each source file, whenever
 you change just one line in one file, you incur that entire
 build time, every time, because it can't just rebuild the one
 source file that changed. You also can't do multi-processor
 builds with all-in-one build strategies.

 4 seconds? That's just untrue. D is actually kinda slow these
 days... In my experience it's slower than modern C++ compilers
 by quite a lot.


 It was definitely about 4 seconds not too long ago, a few years
 at most.

 It does seem to take more time now; I wonder why.


100 seconds is a lot more than 4... 25x even, that's a pretty big
productivity decline ;)
Nov 07 2018
prev sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 11/7/2018 10:08 PM, Vladimir Panteleev wrote:

 It does seem to take more time now; I wonder why.


That's easy. It's because nobody has profiled it for years.

Another reason is the backend being in D now, it is being compiled with DMD 
rather than gcc/clang.
Nov 08 2018
prev sibling next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 8 November 2018 at 04:16:44 UTC, Manu wrote:

 On Tue, Nov 6, 2018 at 10:05 AM Vladimir Panteleev via 
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> 
 wrote:

 [...]


 "Indeed, a clean build of DMD itself (about 170’000 lines of D 
 and 120’000 lines of C/C++) takes no longer than 4 seconds to 
 build on a rather average developer machine."

 ...what!? DMD takes me... (compiling) ... 1 minute 40 seconds 
 to build! And because DMD does all-files-at-once compilation, 
 rather than separate compilation for each source file, whenever 
 you change just one line in one file, you incur that entire 
 build time, every time, because it can't just rebuild the one 
 source file that changed. You also can't do multi-processor 
 builds with all-in-one build strategies.

 4 seconds? That's just untrue. D is actually kinda slow these 
 days... In my experience it's slower than modern C++ compilers 
 by quite a lot.


It sounds like you're not using "a rather average developer 
machine" then, as there's no way DMD should be that slow to build 
on a core i5 or better:

https://forum.dlang.org/post/rqukhkpxcvgiefrdcvdq forum.dlang.org
Nov 07 2018
parent reply Manu <turkeyman gmail.com> writes:
On Wed, Nov 7, 2018 at 10:30 PM Joakim via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 On Thursday, 8 November 2018 at 04:16:44 UTC, Manu wrote:

 On Tue, Nov 6, 2018 at 10:05 AM Vladimir Panteleev via
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com>
 wrote:

 [...]


 "Indeed, a clean build of DMD itself (about 170’000 lines of D
 and 120’000 lines of C/C++) takes no longer than 4 seconds to
 build on a rather average developer machine."

 ...what!? DMD takes me... (compiling) ... 1 minute 40 seconds
 to build! And because DMD does all-files-at-once compilation,
 rather than separate compilation for each source file, whenever
 you change just one line in one file, you incur that entire
 build time, every time, because it can't just rebuild the one
 source file that changed. You also can't do multi-processor
 builds with all-in-one build strategies.

 4 seconds? That's just untrue. D is actually kinda slow these
 days... In my experience it's slower than modern C++ compilers
 by quite a lot.


 It sounds like you're not using "a rather average developer
 machine" then, as there's no way DMD should be that slow to build
 on a core i5 or better:

 https://forum.dlang.org/post/rqukhkpxcvgiefrdcvdq forum.dlang.org


I'm on an i7 with 8 threads and plenty of ram... although threads are
useless, since DMD only uses one ;)
Nov 07 2018
next sibling parent reply Joakim <dlang joakim.fea.st> writes:
On Thursday, 8 November 2018 at 07:41:58 UTC, Manu wrote:

 On Wed, Nov 7, 2018 at 10:30 PM Joakim via 
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> 
 wrote:

 On Thursday, 8 November 2018 at 04:16:44 UTC, Manu wrote:

 On Tue, Nov 6, 2018 at 10:05 AM Vladimir Panteleev via 
 Digitalmars-d-announce 
 <digitalmars-d-announce puremagic.com> wrote:

 [...]


 "Indeed, a clean build of DMD itself (about 170’000 lines of 
 D and 120’000 lines of C/C++) takes no longer than 4 seconds 
 to build on a rather average developer machine."

 ...what!? DMD takes me... (compiling) ... 1 minute 40 
 seconds to build! And because DMD does all-files-at-once 
 compilation, rather than separate compilation for each 
 source file, whenever you change just one line in one file, 
 you incur that entire build time, every time, because it 
 can't just rebuild the one source file that changed. You 
 also can't do multi-processor builds with all-in-one build 
 strategies.

 4 seconds? That's just untrue. D is actually kinda slow 
 these days... In my experience it's slower than modern C++ 
 compilers by quite a lot.


 It sounds like you're not using "a rather average developer 
 machine" then, as there's no way DMD should be that slow to 
 build on a core i5 or better:

 https://forum.dlang.org/post/rqukhkpxcvgiefrdcvdq forum.dlang.org


 I'm on an i7 with 8 threads and plenty of ram... although 
 threads are useless, since DMD only uses one ;)


Running Windows XP? ;) That does sound like Windows though, as I 
do remember being surprised how long dmd took to build on Win7 
when I tried it 8-9 years back. I still don't think the toolchain 
should be _that_ much slower than linux though.

Btw, the extra cores are _not_ useless for the DMD backend, which 
has always used separate compilation, whether written in C++ or D.
Nov 07 2018
parent Manu <turkeyman gmail.com> writes:
On Wed, Nov 7, 2018 at 11:55 PM Joakim via Digitalmars-d-announce
<digitalmars-d-announce puremagic.com> wrote:

 On Thursday, 8 November 2018 at 07:41:58 UTC, Manu wrote:

 On Wed, Nov 7, 2018 at 10:30 PM Joakim via
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com>
 wrote:

 On Thursday, 8 November 2018 at 04:16:44 UTC, Manu wrote:

 On Tue, Nov 6, 2018 at 10:05 AM Vladimir Panteleev via
 Digitalmars-d-announce
 <digitalmars-d-announce puremagic.com> wrote:

 [...]


 "Indeed, a clean build of DMD itself (about 170’000 lines of
 D and 120’000 lines of C/C++) takes no longer than 4 seconds
 to build on a rather average developer machine."

 ...what!? DMD takes me... (compiling) ... 1 minute 40
 seconds to build! And because DMD does all-files-at-once
 compilation, rather than separate compilation for each
 source file, whenever you change just one line in one file,
 you incur that entire build time, every time, because it
 can't just rebuild the one source file that changed. You
 also can't do multi-processor builds with all-in-one build
 strategies.

 4 seconds? That's just untrue. D is actually kinda slow
 these days... In my experience it's slower than modern C++
 compilers by quite a lot.


 It sounds like you're not using "a rather average developer
 machine" then, as there's no way DMD should be that slow to
 build on a core i5 or better:

 https://forum.dlang.org/post/rqukhkpxcvgiefrdcvdq forum.dlang.org


 I'm on an i7 with 8 threads and plenty of ram... although
 threads are useless, since DMD only uses one ;)


 Running Windows XP? ;) That does sound like Windows though, as I
 do remember being surprised how long dmd took to build on Win7
 when I tried it 8-9 years back. I still don't think the toolchain
 should be _that_ much slower than linux though.

 Btw, the extra cores are _not_ useless for the DMD backend, which
 has always used separate compilation, whether written in C++ or D.


No, you're right, the backend builds in 2-3 seconds.
Nov 07 2018
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 11/7/2018 11:41 PM, Manu wrote:

 I'm on an i7 with 8 threads and plenty of ram... although threads are
 useless, since DMD only uses one ;)


So does every other compiler.

To do a multicore build, you'll need to use a makefile that supports -j.
Nov 08 2018
parent Manu <turkeyman gmail.com> writes:
On Thu, Nov 8, 2018 at 12:10 AM Walter Bright via
Digitalmars-d-announce <digitalmars-d-announce puremagic.com> wrote:

 On 11/7/2018 11:41 PM, Manu wrote:

 I'm on an i7 with 8 threads and plenty of ram... although threads are
 useless, since DMD only uses one ;)


 So does every other compiler.

 To do a multicore build, you'll need to use a makefile that supports -j.


Right.
So...?

Also, MSBuild is what people use on Windows... but same applies.
Nov 08 2018
prev sibling next sibling parent Atila Neves <atila.neves gmail.com> writes:
On Thursday, 8 November 2018 at 04:16:44 UTC, Manu wrote:

 On Tue, Nov 6, 2018 at 10:05 AM Vladimir Panteleev via 
 Digitalmars-d-announce <digitalmars-d-announce puremagic.com> 
 wrote:

 This is a tool + article I wrote in February, but never got 
 around to finishing / publishing until today.

 https://blog.thecybershadow.net/2018/02/07/dmdprof/

 Hopefully someone will find it useful.


 "Indeed, a clean build of DMD itself (about 170’000 lines of D 
 and 120’000 lines of C/C++) takes no longer than 4 seconds to 
 build on a rather average developer machine."

 ...what!? DMD takes me... (compiling) ... 1 minute 40 seconds 
 to build! And because DMD does all-files-at-once compilation, 
 rather than separate compilation for each source file, whenever 
 you change just one line in one file, you incur that entire 
 build time, every time, because it can't just rebuild the one 
 source file that changed. You also can't do multi-processor 
 builds with all-in-one build strategies.

 4 seconds? That's just untrue. D is actually kinda slow these 
 days... In my experience it's slower than modern C++ compilers 
 by quite a lot.


Everything on Windows is slower. I'm 99% sure the fault lies with 
NTFS.

I worked on a cross platform framework back in the day, nearly 
all of the code was identical between Linux and Windows. On the 
same dual-boot machine the build was between one and two orders 
of magnitude slower on Windows. Again: same code, same machine, 
no VMs. _Anything_ that used the filesystem was significantly 
slower, including CMake. It got to the point that the devs that 
preferred to use Windows only ever used the unity build because 
incremental ones were too slow to bear.

I'd love to know how long it takes to build clang on your machine.
Nov 08 2018
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2018-11-08 05:16, Manu wrote:


 4 seconds? That's just untrue. D is actually kinda slow these days...
 In my experience it's slower than modern C++ compilers by quite a lot.


This is my result on macOS:

$ $ make -f posix.mak clean
$ time make -f posix.mak -j 16
real	0m3.127s
user	0m5.478s
sys	0m1.686s

-- 
/Jacob Carlborg
Nov 08 2018
next sibling parent "H. S. Teoh" <hsteoh quickfur.ath.cx> writes:
On Thu, Nov 08, 2018 at 06:25:06PM +0100, Jacob Carlborg via
Digitalmars-d-announce wrote:

 On 2018-11-08 05:16, Manu wrote:
 

 4 seconds? That's just untrue. D is actually kinda slow these
 days...  In my experience it's slower than modern C++ compilers by
 quite a lot.


 
 This is my result on macOS:
 
 $ $ make -f posix.mak clean
 $ time make -f posix.mak -j 16
 real	0m3.127s
 user	0m5.478s
 sys	0m1.686s


[...]

Result on Debian/Linux (amd64):

real    0m8.445s
user    0m11.088s
sys     0m1.453s


Slower than C++ compilers?! That's impossible.  There must be something
wrong with your setup, or else with your OS.  Dmd is easily one of the
fastest compilers I've ever used, even after the noticeable slowdown
when we started bootstrapping from D. G++, for example, is at least an
order of magnitude slower.  On my system, anyway. YMMV obviously.

(Of course, it depends on what D features you use... template-heavy and
CTFE-heavy code tends to slow it down pretty badly. But still, it's
pretty fast compared to g++.)


T

-- 
Who told you to swim in Crocodile Lake without life insurance??
Nov 08 2018
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2018-11-08 18:25, Jacob Carlborg wrote:


 This is my result on macOS:
 
 $ $ make -f posix.mak clean
 $ time make -f posix.mak -j 16
 real    0m3.127s
 user    0m5.478s
 sys    0m1.686s


21 seconds on a Windows 10 virtual machine compiling using the win32.mak 
file.

-- 
/Jacob Carlborg
Nov 08 2018
parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Thursday, 8 November 2018 at 19:07:32 UTC, Jacob Carlborg 
wrote:

 21 seconds on a Windows 10 virtual machine compiling using the 
 win32.mak file.


Sounds like we're narrowing it down to the Visual Studio solution.
Nov 08 2018
next sibling parent John Chapman <johnch_atms hotmail.com> writes:
On Thursday, 8 November 2018 at 20:19:47 UTC, Vladimir Panteleev 
wrote:

 On Thursday, 8 November 2018 at 19:07:32 UTC, Jacob Carlborg 
 wrote:

 21 seconds on a Windows 10 virtual machine compiling using the 
 win32.mak file.


 Sounds like we're narrowing it down to the Visual Studio 
 solution.


Well, I'm just using win32.mak in a command prompt and it takes 
~1m 40s to build DMD. It just seems to do nothing for most of 
that time.
Nov 08 2018
prev sibling parent reply kinke <noone nowhere.com> writes:
On Thursday, 8 November 2018 at 20:19:47 UTC, Vladimir Panteleev 
wrote:

 Sounds like we're narrowing it down to the Visual Studio 
 solution.


Mildly interested, I gave DMD master a shot with my 5-years old 
i5-3550 ( 4 GHz) in VS 2017 (+ a recent Visual D beta):

DMD v2.083.0-beta1 host compiler
---
Win32-Debug:      12 secs
Win32-Release: 1m 58 secs

LDC v1.13.0-beta1 host compiler
---
Win32-Debug:      22 secs
Win32-Release: 1m 28 secs

DMD's `-inline` is known to be *slow*.
Nov 08 2018
parent Rainer Schuetze <r.sagitario gmx.de> writes:
On 09/11/2018 00:12, kinke wrote:

 On Thursday, 8 November 2018 at 20:19:47 UTC, Vladimir Panteleev wrote:

 Sounds like we're narrowing it down to the Visual Studio solution.


 
 Mildly interested, I gave DMD master a shot with my 5-years old i5-3550
 ( 4 GHz) in VS 2017 (+ a recent Visual D beta):
 
 DMD v2.083.0-beta1 host compiler
 ---
 Win32-Debug:      12 secs
 Win32-Release: 1m 58 secs
 
 LDC v1.13.0-beta1 host compiler
 ---
 Win32-Debug:      22 secs
 Win32-Release: 1m 28 secs
 


Similar numbers for me too. Please note that the auto-tester shows
similar values for release builds on all platforms, so I suspect very
low numbers reported here are debug build timings.


 DMD's `-inline` is known to be *slow*.


Pedantically, it's not the inliner by itself, but the optimizer that
cannot cope with large functions. With some improved inlining
capabilities that got worse over time.

Visual Studio only provides parallel compilation of projects out of the
box, you have to add it yourself for your favorite language for
parallelization within a project. Igor has done that for the msbuild
integration of dmd and LDC (https://github.com/dlang/visuald/pull/94),
but the PR has kind of stalled due to some undesirable side effects.

Unfortunately, single file compilation is pretty slow, so there is not
so much gain from switching from building per package to single file
parallel compilation unless you have many cores.
Nov 08 2018
prev sibling parent reply Basile B. <b2.temp gmx.com> writes:
On Tuesday, 6 November 2018 at 18:00:22 UTC, Vladimir Panteleev 
wrote:

 This is a tool + article I wrote in February, but never got 
 around to finishing / publishing until today.

 https://blog.thecybershadow.net/2018/02/07/dmdprof/

 Hopefully someone will find it useful.


I put a start to the project then today i remembered callgrind...
What are the pros and cons in comparison with callgrind ? By 
coincidence i had to callgrind dmd today and got answers to a 
performance issue in a very straight forward way because for 
example:

     $ valgrind --tool=callgrind generated/debug/dmd somefile.d

is very simple procedure and produce a file that's easy to 
interpret in kcachegrind (which can then also produce a graph in 
ps format). Note that i don't try to discredit your work, i'm 
just curious to know.
Nov 17 2018
parent reply Vladimir Panteleev <thecybershadow.lists gmail.com> writes:
On Saturday, 17 November 2018 at 23:01:23 UTC, Basile B. wrote:

 I put a start to the project then today i remembered 
 callgrind...
 What are the pros and cons in comparison with callgrind ? By 
 coincidence i had to callgrind dmd today and got answers to a 
 performance issue in a very straight forward way because for 
 example:

     $ valgrind --tool=callgrind generated/debug/dmd somefile.d

 is very simple procedure and produce a file that's easy to 
 interpret in kcachegrind (which can then also produce a graph 
 in ps format). Note that i don't try to discredit your work, 
 i'm just curious to know.


It sounds like you misunderstood the purpose of dmdprof.

dmdprof will show you a profile graph of the source code *being 
compiled* (i.e. which parts take more time to compile and which 
parts cause other parts to get compiled), not of DMD source code.
Nov 17 2018
parent reply Basile B. <b2.temp gmx.com> writes:
On Sunday, 18 November 2018 at 01:12:04 UTC, Vladimir Panteleev 
wrote:

 On Saturday, 17 November 2018 at 23:01:23 UTC, Basile B. wrote:

 I put a start to the project then today i remembered 
 callgrind...
 What are the pros and cons in comparison with callgrind ? By 
 coincidence i had to callgrind dmd today and got answers to a 
 performance issue in a very straight forward way because for 
 example:

     $ valgrind --tool=callgrind generated/debug/dmd somefile.d

 is very simple procedure and produce a file that's easy to 
 interpret in kcachegrind (which can then also produce a graph 
 in ps format). Note that i don't try to discredit your work, 
 i'm just curious to know.


 It sounds like you misunderstood the purpose of dmdprof.

 dmdprof will show you a profile graph of the source code *being 
 compiled* (i.e. which parts take more time to compile and which 
 parts cause other parts to get compiled), not of DMD source 
 code.


Damn, you're right. Actually i didn't try your stuff at all and 
instead used callgrind intuitively. Bad advice was here: 
https://github.com/dlang/dmd/pull/8945#issuecomment-439388332. 
Also along the thread there was discussion about building DMD and 
one saying that DMD had not been profiled for ages.
Nov 17 2018
parent Nicholas Wilson <iamthewilsonator hotmail.com> writes:
On Sunday, 18 November 2018 at 04:38:57 UTC, Basile B. wrote:

 Damn, you're right. Actually i didn't try your stuff at all and 
 instead used callgrind intuitively. Bad advice was here: 
 https://github.com/dlang/dmd/pull/8945#issuecomment-439388332. 
 Also along the thread there was discussion about building DMD 
 and one saying that DMD had not been profiled for ages.


My bad.
Nov 18 2018