www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - [Help Needed] - Debugging compilation time

reply Hipreme <msnmancini hotmail.com> writes:
Hey guys, I have been complaining a lot of time right now from D 
compilation speed at least for my project.

I have:
- Underused CTFE
- Underused Templates
- Avoided importing standard libraries
- Created a multi module projects for better code reuse

Those are all the techniques I have tried to maintain my 
compilation times lower, yet, It still builds slow even when 
using the --combined (which does a single compiler run on dub)

So, doesn't matter if all is imported at once or not, it seems 
the problem seems to be somewhere in my project which I've been 
unable to find. I wanted to show someone who is more experienced 
on the time trace from LDC to check what I've been doing wrong.

This time trace took 9 seconds. DMD takes 7 seconds to build my 
project. Adam has built his entire arsd in 2.5 seconds, while my 
PC is faster and arsd is much bigger than my project, this does 
not make any sense. So I would gladly wish you guys help:

My repository: https://github.com/MrcSnm/HipremeEngine/
My time trace: https://ufile.io/gmvw1wlu (This time trace will be 
in air for 30 days only)
Oct 21 2022
next sibling parent "H. S. Teoh" <hsteoh qfbox.info> writes:
On Fri, Oct 21, 2022 at 04:32:17PM +0000, Hipreme via Digitalmars-d-learn wrote:

 Hey guys, I have been complaining a lot of time right now from D
 compilation speed at least for my project.
 
 I have:
 - Underused CTFE
 - Underused Templates
 - Avoided importing standard libraries
 - Created a multi module projects for better code reuse
 
 Those are all the techniques I have tried to maintain my compilation
 times lower, yet, It still builds slow even when using the --combined
 (which does a single compiler run on dub)


Using dub immediately slows you down by at least several seconds. There
may be some options to skip the time-consuming dependency graph
resolution and network access.  But my personal preference is to use an
offline build system with less overhead.



 So, doesn't matter if all is imported at once or not, it seems the
 problem seems to be somewhere in my project which I've been unable to
 find. I wanted to show someone who is more experienced on the time
 trace from LDC to check what I've been doing wrong.


LDC generally tends to be quite a bit slower than DMD because of the
time spent in aggressive optimizations.  For development builds,
consider using DMD instead.



 This time trace took 9 seconds. DMD takes 7 seconds to build my
 project.  Adam has built his entire arsd in 2.5 seconds, while my PC
 is faster and arsd is much bigger than my project, this does not make
 any sense.


It could be caused by a number of different things:

- Using many nested templates will slow down compilation.
- Doing excessive CTFE computations will slow things down.
- Using unusually-large static arrays in some cases may trigger slow
  paths in the front-end, consuming lots of compiler memory.
- Using string imports may slow things down as the compiler parses the
  imported file(s).
- Having excessively-large function bodies may trigger some O(n^2) paths
  in the compiler that significantly slows things down.
- Using excessive mixins may also slow things down due to copying of the
  AST.

OTOH, uninstantiated templates are cheap (the only cost is parsing them,
which is very fast): just avoiding templates along may not save you much
time if it's being spent elsewhere.  I suspect a good chunk of Adam's
code is in uninstantiated templates, so a straight comparison isn't
really fair, if those templates aren't actually being instantiated in
the first place.

I also notice you use mixin templates a lot; it might be worth
investigating whether they might be the cause of your issue.



 So I would gladly wish you guys help:
 
 My repository: https://github.com/MrcSnm/HipremeEngine/
 My time trace: https://ufile.io/gmvw1wlu (This time trace will be in air for
 30 days only)


Sorry, no time to look into it in detail, but I did skim over a few
files and note that you use mixin templates a lot.  I haven't had much
experience with mixin templates but it might be worth investigating
whether they might be causing compile time slowdowns.


T

-- 
They say that "guns don't kill people, people kill people." Well I think the
gun helps. If you just stood there and yelled BANG, I don't think you'd kill
too many people. -- Eddie Izzard, Dressed to Kill
Oct 21 2022
prev sibling next sibling parent reply ryuukk_ <ryuukk.dev gmail.com> writes:
I tried your project:

Linux x64

```
git clone https://github.com/MrcSnm/HipremeEngine.git
cd HipremeEngine
dub build (once to download dependencies if any)
time dub build -f


real	0m4.604s
user	0m3.686s
sys	0m0.900s
```

4.6 sec for a FULL rebuild doesn't seem that bad

and

```
real	0m1.730s
user	0m1.480s
sys	0m0.245s

```

after editing one module, not bad at all

As other people say:

LDC for release with optimizations
DMD for development

THat's what i personally do
Oct 21 2022
parent reply Hipreme <msnmancini hotmail.com> writes:
On Friday, 21 October 2022 at 18:10:39 UTC, ryuukk_ wrote:

 I tried your project:

 Linux x64

 ```
 git clone https://github.com/MrcSnm/HipremeEngine.git
 cd HipremeEngine
 dub build (once to download dependencies if any)
 time dub build -f


 real	0m4.604s
 user	0m3.686s
 sys	0m0.900s
 ```

 4.6 sec for a FULL rebuild doesn't seem that bad

 and

 ```
 real	0m1.730s
 user	0m1.480s
 sys	0m0.245s

 ```

 after editing one module, not bad at all

 As other people say:

 LDC for release with optimizations
 DMD for development

 THat's what i personally do



I'm not using anti virus :c

Seems that Linux really is significantly faster for building , 
1.7secs for a rebuild seems pretty nice for me, though I can't 
say it happens the same on my ground
Oct 22 2022
parent ryuukk_ <ryuukk.dev gmail.com> writes:
On Saturday, 22 October 2022 at 12:27:21 UTC, Hipreme wrote:

 On Friday, 21 October 2022 at 18:10:39 UTC, ryuukk_ wrote:

 I tried your project:

 Linux x64

 ```
 git clone https://github.com/MrcSnm/HipremeEngine.git
 cd HipremeEngine
 dub build (once to download dependencies if any)
 time dub build -f


 real	0m4.604s
 user	0m3.686s
 sys	0m0.900s
 ```

 4.6 sec for a FULL rebuild doesn't seem that bad

 and

 ```
 real	0m1.730s
 user	0m1.480s
 sys	0m0.245s

 ```

 after editing one module, not bad at all

 As other people say:

 LDC for release with optimizations
 DMD for development

 THat's what i personally do



 I'm not using anti virus :c

 Seems that Linux really is significantly faster for building , 
 1.7secs for a rebuild seems pretty nice for me, though I can't 
 say it happens the same on my ground


What exactly takes time? do you notice a specific module taking 
longer than others? or is it the linker?

A module that is windows specific might be causing the issue you 
are having

Try to output the mixin file and see what generates lot of code: 
https://dlang.org/dmd-linux.html#switch-mixin

I'll see if i can test on Windows later today
Oct 22 2022
prev sibling next sibling parent ryuukk_ <ryuukk.dev gmail.com> writes:
Make sure you have the latest version of DMD

Make sure your antivirus isn't blocking your files (scanning), 
it's a common thing with Windows, whitelist dmd folder, your dev 
folder and dub folder
Oct 21 2022
prev sibling parent apz28 <home home.com> writes:
On Friday, 21 October 2022 at 16:32:17 UTC, Hipreme wrote:

 Hey guys, I have been complaining a lot of time right now from 
 D compilation speed at least for my project.

 I have:
 - Underused CTFE
 - Underused Templates
 - Avoided importing standard libraries
 - Created a multi module projects for better code reuse


The slow is from executing CTFE (template constraints, enum 
expression, static if ...). The culprit is this function:  
Expression.ctfeInterpret() in this module dmd.expression.d
Even for compiling DMD, that function will take 1/3 of total times

Try to reduce CTFE usage than it will be fast again. D should 
analyze to make executing CTFE faster

Cheers
Oct 21 2022