www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - DMD release compiler flags when building with LDC

reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
Have anybody experimented with release compilation flags when 
building dmd with ldc?

I'm currently calling make as

make -f posix.mak \
     DFLAGS="-noboundscheck" \
     ENABLE_RELEASE=1 \
     ENABLE_LTO=1 \
     HOST_CXX=g++ \
     HOST_DMD=ldmd2

And I measure about a 20-25% drop in runtime compared to the 
standard released version of dmd.

Is profile guided optimization (PGO) worth a try?
Oct 22
next sibling parent reply kinke <noone nowhere.com> writes:
On Tuesday, 22 October 2019 at 14:02:28 UTC, Per Nordlöw wrote:
     ENABLE_LTO=1 \
This has exactly 0 effect on DMD itself, from DMD's src/posix.mak: ifdef ENABLE_LTO CXXFLAGS += -flto endif
Oct 22
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 22 October 2019 at 18:55:03 UTC, kinke wrote:
 On Tuesday, 22 October 2019 at 14:02:28 UTC, Per Nordlöw wrote:
     ENABLE_LTO=1 \
This has exactly 0 effect on DMD itself, from DMD's src/posix.mak: ifdef ENABLE_LTO CXXFLAGS += -flto endif
So, I presume we could set ifdef ENABLE_LTO DFLAGS += -flto=full endif in the case when `HOST_DC` is either `ldmd2` or `ldc2` then, right? This works for me.
Oct 23
next sibling parent reply Daniel Kozak <kozzi11 gmail.com> writes:
On Wed, Oct 23, 2019 at 11:50 AM Per Nordlöw via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Tuesday, 22 October 2019 at 18:55:03 UTC, kinke wrote:
 On Tuesday, 22 October 2019 at 14:02:28 UTC, Per Nordlöw wrote:
     ENABLE_LTO=1 \
This has exactly 0 effect on DMD itself, from DMD's src/posix.mak: ifdef ENABLE_LTO CXXFLAGS += -flto endif
So, I presume we could set ifdef ENABLE_LTO DFLAGS += -flto=full endif in the case when `HOST_DC` is either `ldmd2` or `ldc2` then, right? This works for me.
https://bugs.archlinux.org/task/63569
Oct 23
parent reply Per =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Wednesday, 23 October 2019 at 10:34:17 UTC, Daniel Kozak wrote:
 This works for me.
https://bugs.archlinux.org/task/63569
Shouldn't +ifeq ($(HOST_DMD_KIND), ldc) be +ifeq ($(HOST_DMD_KIND), ldc2) ?
Oct 23
next sibling parent Seb <seb wilzba.ch> writes:
On Wednesday, 23 October 2019 at 11:03:18 UTC, Per Nordlöw wrote:
 On Wednesday, 23 October 2019 at 10:34:17 UTC, Daniel Kozak 
 wrote:
 This works for me.
https://bugs.archlinux.org/task/63569
Shouldn't +ifeq ($(HOST_DMD_KIND), ldc) be +ifeq ($(HOST_DMD_KIND), ldc2) ?
ArchLinux ships LDC as ldc and ldc2. https://github.com/ldc-developers/ldc/issues/2701
Oct 23
prev sibling parent Daniel Kozak <kozzi11 gmail.com> writes:
On Wed, Oct 23, 2019 at 1:05 PM Per Nordlöw via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
 On Wednesday, 23 October 2019 at 10:34:17 UTC, Daniel Kozak wrote:
 This works for me.
https://bugs.archlinux.org/task/63569
Shouldn't +ifeq ($(HOST_DMD_KIND), ldc) be +ifeq ($(HOST_DMD_KIND), ldc2) ?
No, if you look to posix.mak you will see this: https://github.com/dlang/dmd/blob/v2.088.1/src/posix.mak#L167 ifneq (,$(findstring ldc,$(HOST_DMD_VERSION))$(findstring LDC,$(HOST_DMD_VERSION))) HOST_DMD_KIND=ldc HOST_DMD_VERNUM=2 endif
Oct 23
prev sibling parent reply kinke <kinke gmx.net> writes:
On Wednesday, 23 October 2019 at 09:49:57 UTC, Per Nordlöw wrote:
 On Tuesday, 22 October 2019 at 18:55:03 UTC, kinke wrote:
 On Tuesday, 22 October 2019 at 14:02:28 UTC, Per Nordlöw wrote:
     ENABLE_LTO=1 \
This has exactly 0 effect on DMD itself, from DMD's src/posix.mak: ifdef ENABLE_LTO CXXFLAGS += -flto endif
So, I presume we could set ifdef ENABLE_LTO DFLAGS += -flto=full endif in the case when `HOST_DC` is either `ldmd2` or `ldc2` then, right? This works for me.
My bad, I thought build.d was a replacement for the Makefile, instead the Makefile is based on it, so this should kick in: https://github.com/dlang/dmd/blob/ce7576f256d4efaff843f5e7d6ed39e8d7f32a4e/src/build.d#L781-L784 druntime is linked as machine code though, so an additional `-defaultlib=druntime-ldc-lto` (if available in the used LDC package, true for almost all official packages) would additionally enable LTO across DMD and druntime.
Oct 23
next sibling parent reply Daniel Kozak <kozzi11 gmail.com> writes:
On Wed, Oct 23, 2019 at 1:25 PM kinke via Digitalmars-d
<digitalmars-d puremagic.com> wrote:
...
 My bad, I thought build.d was a replacement for the Makefile,
 instead the Makefile is based on it, so this should kick in:
 https://github.com/dlang/dmd/blob/ce7576f256d4efaff843f5e7d6ed39e8d7f32a4e/src/build.d#L781-L784

 druntime is linked as machine code though, so an additional
 `-defaultlib=druntime-ldc-lto` (if available in the used LDC
 package, true for almost all official packages) would
 additionally enable LTO across DMD and druntime.
Not exactly, build.d has been use only for some part of dmd, so this is reason why I have opened https://bugs.archlinux.org/task/63569. There has been mismatch (some code has been compiled witl LTO and some was not ). It is possible right now build.d is used for everything so it could work.
Oct 23
parent kinke <noone nowhere.com> writes:
On Wednesday, 23 October 2019 at 11:29:11 UTC, Daniel Kozak wrote:
 Not exactly, build.d has been use only for some part of dmd, so 
 this is reason why I have opened 
 https://bugs.archlinux.org/task/63569. There has been mismatch 
 (some code has been compiled witl LTO and some was not ). It is 
 possible right now build.d is used for everything so it could 
 work.
It looked like it was all compiled with build.d, but I only glanced over it. Mixing bitcode and machine code objects should be fine; archiving a mix into static libraries too, but that requires archiving with LDC (`-lib`).
Oct 23
prev sibling parent Jonathan Marler <johnnymarler gmail.com> writes:
On Wednesday, 23 October 2019 at 11:24:32 UTC, kinke wrote:
 On Wednesday, 23 October 2019 at 09:49:57 UTC, Per Nordlöw 
 wrote:
 On Tuesday, 22 October 2019 at 18:55:03 UTC, kinke wrote:
 On Tuesday, 22 October 2019 at 14:02:28 UTC, Per Nordlöw 
 wrote:
     ENABLE_LTO=1 \
This has exactly 0 effect on DMD itself, from DMD's src/posix.mak: ifdef ENABLE_LTO CXXFLAGS += -flto endif
So, I presume we could set ifdef ENABLE_LTO DFLAGS += -flto=full endif in the case when `HOST_DC` is either `ldmd2` or `ldc2` then, right? This works for me.
My bad, I thought build.d was a replacement for the Makefile, instead the Makefile is based on it, so this should kick in: https://github.com/dlang/dmd/blob/ce7576f256d4efaff843f5e7d6ed39e8d7f32a4e/src/build.d#L781-L784 druntime is linked as machine code though, so an additional `-defaultlib=druntime-ldc-lto` (if available in the used LDC package, true for almost all official packages) would additionally enable LTO across DMD and druntime.
build.d is meant to replace the Makefiles. It just takes a long time to get PRs through and they need to be small. So to make the change incremental I've been removing logic from the Makefiles and forwarding the targets to call build.d instead. We've recently made some good progress and we just started getting help from MoonlightSentinel so hopefully well get it all moved over soon. That being said, having the Makefiles around doesn't really bother me. Maybe we'll leave them so people can still use make to build dmd. So long as all the logic is removed and all it does is forward to build.d.
Oct 24
prev sibling parent Seb <seb wilzba.ch> writes:
On Tuesday, 22 October 2019 at 14:02:28 UTC, Per Nordlöw wrote:
 Have anybody experimented with release compilation flags when 
 building dmd with ldc?
 ...
 And I measure about a 20-25% drop in runtime compared to the 
 standard released version of dmd.
Last time I checked it was about 40-50% (depends a bit on what you use to test, your machine etc.). So, yup the released binaries could be easily twice as fast. This is well known. The problem is that no one except Martin has a full running setup of the release pipeline: https://github.com/dlang/installer Hence, this change hasn't happened. IMHO, we should build them directly in the respective CIs like e.g. LDC does, because then anyone can easily modify and improve the setup.
 Is profile guided optimization (PGO) worth a try?
Yes.
Oct 23