• Andrei Alexandrescu (3/3) Jun 10 2015 https://issues.dlang.org/show_bug.cgi?id=14679
• weaselcat (5/8) Jun 10 2015 dmd's build times are hurt by its current allocation strategy,
• Tofu Ninja (2/5) Jun 10 2015 I think I read someone say that ddmd doesn't use the GC. Though I
• Jacob Carlborg (4/6) Jun 10 2015 Doesn't the GC need to do some locking when allocating?
• David Nadlinger (5/8) Jun 10 2015 DDMD currently runs with the GC disabled, as it would crash
• Jacob Carlborg (4/7) Jun 11 2015 But it allocates using the GC?
• David Nadlinger (8/9) Jun 11 2015 Depends. The DMD-built one forgoes the GC for the
• Yuxuan Shui (5/8) Jun 10 2015 As for the .di file, I believe Rust compiler store serialized AST
• Dmitry Olshansky (7/10) Jun 11 2015 Not processing templates is cool idea.
• Jonathan M Davis (27/36) Jun 11 2015 Yeah. Pretty much any and all performance improvements that we
• Laeeth Isharc (8/18) Jun 11 2015 I am following along and learning, but could you perhaps elaborate
• Jonathan M Davis (26/47) Jun 11 2015 It's what happens when you use traits like isInputRange,
• Laeeth Isharc (3/6) Jun 12 2015 Thank you!
• Andrei Alexandrescu (7/9) Jun 11 2015 I did some preliminary investigation, with promising results. I think we...
• Andrei Alexandrescu (8/16) Jun 11 2015 Forgot to mention: I submitted three more compiler bug reports, ranging
• Andrei Alexandrescu (5/9) Jun 11 2015 And one more, which is pretty crucial to the entire approach to headers
• weaselcat (5/14) Jun 11 2015 Did you see any performance improvements?
• Andrei Alexandrescu (6/23) Jun 11 2015 Yes, refer to measurements in

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

https://issues.dlang.org/show_bug.cgi?id=14679
https://issues.dlang.org/show_bug.cgi?id=14680

Andrei

"weaselcat" <weaselcat gmail.com> writes:

On Thursday, 11 June 2015 at 00:50:01 UTC, Andrei Alexandrescu 
wrote:
 https://issues.dlang.org/show_bug.cgi?id=14679
 https://issues.dlang.org/show_bug.cgi?id=14680

 Andrei

dmd's build times are hurt by its current allocation strategy, 
tons of pagefaults. I'm actually surprised that ddmd isn't 
running circles around it by virtue of having a GC.

"Tofu Ninja" <emmons0 purdue.edu> writes:

 dmd's build times are hurt by its current allocation strategy, 
 tons of pagefaults. I'm actually surprised that ddmd isn't 
 running circles around it by virtue of having a GC.

I think I read someone say that ddmd doesn't use the GC. Though I 
definitely could be wrong.

Jacob Carlborg <doob me.com> writes:

On 2015-06-11 03:45, Tofu Ninja wrote:

 I think I read someone say that ddmd doesn't use the GC. Though I
 definitely could be wrong.

Doesn't the GC need to do some locking when allocating?

-- 
/Jacob Carlborg

"David Nadlinger" <code klickverbot.at> writes:

On Thursday, 11 June 2015 at 01:43:22 UTC, weaselcat wrote:
 dmd's build times are hurt by its current allocation strategy, 
 tons of pagefaults. I'm actually surprised that ddmd isn't 
 running circles around it by virtue of having a GC.

DDMD currently runs with the GC disabled, as it would crash 
otherwise (probably because the only reference to an object is 
held by C++ code sometimes).

  - David

Jacob Carlborg <doob me.com> writes:

On 2015-06-11 08:32, David Nadlinger wrote:

 DDMD currently runs with the GC disabled, as it would crash otherwise
 (probably because the only reference to an object is held by C++ code
 sometimes).

But it allocates using the GC?

-- 
/Jacob Carlborg

"David Nadlinger" <code klickverbot.at> writes:

On Thursday, 11 June 2015 at 07:07:34 UTC, Jacob Carlborg wrote:
 But it allocates using the GC?

Depends. The DMD-built one forgoes the GC for the 
bump-the-pointer allocator by providing alternate implementations 
for the relevant druntime functions (_d_newclass and so on).

However, Daniel and I disabled this in the LDC build for 
unrelated reasons, and it did not seem to make any difference at 
all performance-wise for building the Phobos unittests.

  - David

"Yuxuan Shui" <yshuiv7 gmail.com> writes:

On Thursday, 11 June 2015 at 00:50:01 UTC, Andrei Alexandrescu 
wrote:
 https://issues.dlang.org/show_bug.cgi?id=14679
 https://issues.dlang.org/show_bug.cgi?id=14680

 Andrei

As for the .di file, I believe Rust compiler store serialized AST 
in .a and .so. Is it possible that we do the same? And will doing 
that improve build time?

Dmitry Olshansky <dmitry.olsh gmail.com> writes:

On 11-Jun-2015 03:50, Andrei Alexandrescu wrote:
 https://issues.dlang.org/show_bug.cgi?id=14679
 https://issues.dlang.org/show_bug.cgi?id=14680

 Andrei

Not processing templates is cool idea.
However the main problem with DMD's build times on moderately sized 
projects is a huge memory leak in the compiler, mostly having to do with 
CTFE.


-- 
Dmitry Olshansky

"Jonathan M Davis" <jmdavisProg gmx.com> writes:

On Thursday, 11 June 2015 at 07:49:35 UTC, Dmitry Olshansky wrote:
 On 11-Jun-2015 03:50, Andrei Alexandrescu wrote:
 https://issues.dlang.org/show_bug.cgi?id=14679
 https://issues.dlang.org/show_bug.cgi?id=14680

 Andrei

 Not processing templates is cool idea.
 However the main problem with DMD's build times on moderately 
 sized projects is a huge memory leak in the compiler, mostly 
 having to do with CTFE.

Yeah. Pretty much any and all performance improvements that we 
can make should be made, and I don't think that they should be 
discouraged unless there's actually something wrong with them. 
But CTFE is the real killer. Improving that would probably 
improve build times by at least an order of magnitude for 
CTFE-heavy builds. Unfortunately, Don hasn't had time to work on 
CTFE for the last year+, and while Daniel has some definite ideas 
of how to improve it, he doesn't want to do that until we're 
fully switched over to D for the compiler, since it'll be easier 
to implement the changes there. So, we're unlikely to see any 
improvements there until we've fully switched to ddmd unless 
another compiler dev decides to take the plunge. At least we're 
getting close to ddmd though.

The other fun one is the insane number of template instantations 
we get for stuff like std.algorithm thanks to all of those helper 
templates like isInputRange. IIRC, std.algorithm's unit tests 
instantiate over a million different template instances - and 
much of that is simply for template constraints and static ifs. 
So, improving build times with regards to templates which are 
instantiated would probably be the second largest gain to be had 
behind CTFE if we could improve it, but obviously, we'd have to 
make such improvements to be sure.

Regardless, the more that we can reasonably do to improve build 
times, the better. And while we want to fix the big problems, 
every little bit helps.

- Jonathan M Davis

"Laeeth Isharc" <nospamlaeeth nospam.laeeth.com> writes:

On Thursday, 11 June 2015 at 21:02:36 UTC, Jonathan M Davis wrote:
 The other fun one is the insane number of template 
 instantations we get for stuff like std.algorithm thanks to all 
 of those helper templates like isInputRange. IIRC, 
 std.algorithm's unit tests instantiate over a million different 
 template instances - and much of that is simply for template 
 constraints and static ifs. So, improving build times with 
 regards to templates which are instantiated would probably be 
 the second largest gain to be had behind CTFE if we could 
 improve it, but obviously, we'd have to make such improvements 
 to be sure.

I am following along and learning, but could you perhaps elaborate
on this a bit more if you have time.  I can see that the unit test
template proliferation makes running unit tests slow for
std.algorithm.  But why does it affect builds in other respects
if you are just importing std.algorithm from a non-phobos code
base?  Or was this an example of the extreme case as an 
illustration?

"Jonathan M Davis" <jmdavisProg gmx.com> writes:

On Thursday, 11 June 2015 at 21:41:27 UTC, Laeeth Isharc wrote:
 On Thursday, 11 June 2015 at 21:02:36 UTC, Jonathan M Davis 
 wrote:
 The other fun one is the insane number of template 
 instantations we get for stuff like std.algorithm thanks to 
 all of those helper templates like isInputRange. IIRC, 
 std.algorithm's unit tests instantiate over a million 
 different template instances - and much of that is simply for 
 template constraints and static ifs. So, improving build times 
 with regards to templates which are instantiated would 
 probably be the second largest gain to be had behind CTFE if 
 we could improve it, but obviously, we'd have to make such 
 improvements to be sure.

 I am following along and learning, but could you perhaps 
 elaborate
 on this a bit more if you have time.  I can see that the unit 
 test
 template proliferation makes running unit tests slow for
 std.algorithm.  But why does it affect builds in other respects
 if you are just importing std.algorithm from a non-phobos code
 base?  Or was this an example of the extreme case as an 
 illustration?

It's what happens when you use traits like isInputRange, 
isForwardRange, etc. all over the place. std.algorithm is likely 
an extreme case, because it's _all_ templated, and it has lots of 
different static if branches and function overloads for 
efficiency, and of course, we're talking about the unit tests in 
particular, so they have to be instantiating all of the various 
functions in std.algorithm with all kinds of different arguments. 
You quickly get a combinatorial explosion of template 
instantiations where a large portion of those templates are 
eponymous templates used specifically for template constraints 
and static ifs.

I'd have to track down the posts that Don made on the matter to 
give any concrete details, but it's was quite surprising to many 
of us when we found out just how many times many of these basic 
trait templates were being instantiated.

User code will run into similar problems if it makes heavy use of 
such templates in template constraints and static if branches and 
then actually tests all the various combinations in its unit 
tests, but your average program likely doesn't instantiate stuff 
like isInputRange anywhere near as much as std.algorithm's unit 
tests do. It will mostly be unit test builds which have this 
problem. Still, anything that we can do to improve the compiler's 
performance in the face of these common idioms will definitely 
help compilation time.

- Jonathan M Davis

"Laeeth Isharc" <laeeth nospamlaeeth.com> writes:

On Thursday, 11 June 2015 at 23:19:06 UTC, Jonathan M Davis wrote:
 It's what happens when you use traits like isInputRange, 
 isForwardRange, etc. all over the place. std.algorithm is 
 likely an extreme case

Thank you!


Laeeth.

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

On 6/10/15 5:50 PM, Andrei Alexandrescu wrote:
 https://issues.dlang.org/show_bug.cgi?id=14679
 https://issues.dlang.org/show_bug.cgi?id=14680

I did some preliminary investigation, with promising results. I think we 
should move forward with this.

As a perk, we'll dogfood the .di generation which is clearly not 
currently usable. That in turn is making separate compilation of 
large-scale projects unnecessarily difficult.


Andrei

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

On 6/11/15 10:05 PM, Andrei Alexandrescu wrote:
 On 6/10/15 5:50 PM, Andrei Alexandrescu wrote:
 https://issues.dlang.org/show_bug.cgi?id=14679
 https://issues.dlang.org/show_bug.cgi?id=14680

 I did some preliminary investigation, with promising results. I think we
 should move forward with this.

 As a perk, we'll dogfood the .di generation which is clearly not
 currently usable. That in turn is making separate compilation of
 large-scale projects unnecessarily difficult.

Forgot to mention: I submitted three more compiler bug reports, ranging 
from trivial to less so:

https://issues.dlang.org/show_bug.cgi?id=14687
https://issues.dlang.org/show_bug.cgi?id=14688
https://issues.dlang.org/show_bug.cgi?id=14689

Any takers please?


Andrei

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

On 6/11/15 10:07 PM, Andrei Alexandrescu wrote:
 https://issues.dlang.org/show_bug.cgi?id=14687
 https://issues.dlang.org/show_bug.cgi?id=14688
 https://issues.dlang.org/show_bug.cgi?id=14689

 Any takers please?

And one more, which is pretty crucial to the entire approach to headers 
and inline functions:

https://issues.dlang.org/show_bug.cgi?id=14690


Andrei

"weaselcat" <weaselcat gmail.com> writes:

On Friday, 12 June 2015 at 05:05:47 UTC, Andrei Alexandrescu 
wrote:
 On 6/10/15 5:50 PM, Andrei Alexandrescu wrote:
 https://issues.dlang.org/show_bug.cgi?id=14679
 https://issues.dlang.org/show_bug.cgi?id=14680

 I did some preliminary investigation, with promising results. I 
 think we should move forward with this.

 As a perk, we'll dogfood the .di generation which is clearly 
 not currently usable. That in turn is making separate 
 compilation of large-scale projects unnecessarily difficult.


 Andrei

Did you see any performance improvements?
I played around with .di files before but had trouble using them 
when building phobos.

Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:

On 6/11/15 10:18 PM, weaselcat wrote:
 On Friday, 12 June 2015 at 05:05:47 UTC, Andrei Alexandrescu wrote:
 On 6/10/15 5:50 PM, Andrei Alexandrescu wrote:
 https://issues.dlang.org/show_bug.cgi?id=14679
 https://issues.dlang.org/show_bug.cgi?id=14680

 I did some preliminary investigation, with promising results. I think
 we should move forward with this.

 As a perk, we'll dogfood the .di generation which is clearly not
 currently usable. That in turn is making separate compilation of
 large-scale projects unnecessarily difficult.


 Andrei

 Did you see any performance improvements?

Yes, refer to measurements in 
https://issues.dlang.org/show_bug.cgi?id=14680. At least casual 
scripting will be sensibly helped.

 I played around with .di files before but had trouble using them when
 building phobos.

Please add to the bug reports I've sent.


Andrei