www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - PGO not only for own code but for D runtime & Phobos too?

reply =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
Did anyone already try to use PGO with his own code and for the D 
runtime and phobos?

Any indication how big the performance difference is when the D runtime 
and Phobos is compiled using PGO?

-- 
Robert M. MŁnch
http://www.saphirion.com
smarter | better | faster
Oct 03
parent reply Jon Degenhardt <jond noreply.com> writes:
On Thursday, 3 October 2019 at 15:15:44 UTC, Robert M. M√ľnch 
wrote:
 Did anyone already try to use PGO with his own code and for the 
 D runtime and phobos?

 Any indication how big the performance difference is when the D 
 runtime and Phobos is compiled using PGO?
I did a study of LTO and PGO on the combined application and libraries (druntime & phobos). It's available here: https://github.com/eBay/tsv-utils/blob/master/docs/lto-pgo-study.md. For the apps tested it's a big win to include druntime and phobos in the LTO/PGO target. PGO makes a difference standalone, though I didn't try to separate the app code from the druntime/phobos code. --Jon
Oct 03
next sibling parent reply Arun Chandrasekaran <aruncxy gmail.com> writes:
On Thursday, 3 October 2019 at 16:01:19 UTC, Jon Degenhardt wrote:
 On Thursday, 3 October 2019 at 15:15:44 UTC, Robert M. M√ľnch 
 wrote:
 Did anyone already try to use PGO with his own code and for 
 the D runtime and phobos?

 Any indication how big the performance difference is when the 
 D runtime and Phobos is compiled using PGO?
I did a study of LTO and PGO on the combined application and libraries (druntime & phobos). It's available here: https://github.com/eBay/tsv-utils/blob/master/docs/lto-pgo-study.md. For the apps tested it's a big win to include druntime and phobos in the LTO/PGO target. PGO makes a difference standalone, though I didn't try to separate the app code from the druntime/phobos code. --Jon
A very interesting and entertaining talk on measuring performance https://www.youtube.com/watch?v=r&feature=youtu.be-TLSBdHe1A I thought only Andrei can make tech talk hilarious and interesting at the same time. This one exceeded my expectations.
Oct 03
parent reply James Blachly <james.blachly gmail.com> writes:
On 10/3/19 6:34 PM, Arun Chandrasekaran wrote:
 On Thursday, 3 October 2019 at 16:01:19 UTC, Jon Degenhardt wrote:
 On Thursday, 3 October 2019 at 15:15:44 UTC, Robert M. M√ľnch wrote:
 Did anyone already try to use PGO with his own code and for the D 
 runtime and phobos?

 Any indication how big the performance difference is when the D 
 runtime and Phobos is compiled using PGO?
I did a study of LTO and PGO on the combined application and libraries (druntime & phobos). It's available here: https://github.com/eBay/tsv-utils/blob/master/docs/lto-pgo-study.md. For the apps tested it's a big win to include druntime and phobos in the LTO/PGO target. PGO makes a difference standalone, though I didn't try to separate the app code from the druntime/phobos code. --Jon
A very interesting and entertaining talk on measuring performance https://www.youtube.com/watch?v=r&feature=youtu.be-TLSBdHe1A I thought only Andrei can make tech talk hilarious and interesting at the same time. This one exceeded my expectations.
"Video unavailable" :-(
Oct 03
next sibling parent reply mipri <mipri minimaltype.com> writes:
On Friday, 4 October 2019 at 00:51:47 UTC, James Blachly wrote:
 On 10/3/19 6:34 PM, Arun Chandrasekaran wrote:
 A very interesting and entertaining talk on measuring 
 performance 
 https://www.youtube.com/watch?v=r&feature=youtu.be-TLSBdHe1A
 
 I thought only Andrei can make tech talk hilarious and 
 interesting at the same time. This one exceeded my 
 expectations.
"Video unavailable" :-(
It's probably https://www.youtube.com/watch?v=r-TLSBdHe1A
Oct 03
parent Jon Degenhardt <jond noreply.com> writes:
On Friday, 4 October 2019 at 01:04:39 UTC, mipri wrote:
 On Friday, 4 October 2019 at 00:51:47 UTC, James Blachly wrote:
 On 10/3/19 6:34 PM, Arun Chandrasekaran wrote:
 A very interesting and entertaining talk on measuring 
 performance 
 https://www.youtube.com/watch?v=r&feature=youtu.be-TLSBdHe1A
 
 I thought only Andrei can make tech talk hilarious and 
 interesting at the same time. This one exceeded my 
 expectations.
"Video unavailable" :-(
It's probably https://www.youtube.com/watch?v=r-TLSBdHe1A
That's a really nice talk, thanks for posting it. For folks who may missed it, Andre's recent cppcon talk is superb, definitely worth viewing. --Jon
Oct 03
prev sibling parent Arun Chandrasekaran <aruncxy gmail.com> writes:
On Friday, 4 October 2019 at 00:51:47 UTC, James Blachly wrote:
 On 10/3/19 6:34 PM, Arun Chandrasekaran wrote:
 On Thursday, 3 October 2019 at 16:01:19 UTC, Jon Degenhardt 
 wrote:
 [...]
A very interesting and entertaining talk on measuring performance https://www.youtube.com/watch?v=r&feature=youtu.be-TLSBdHe1A I thought only Andrei can make tech talk hilarious and interesting at the same time. This one exceeded my expectations.
"Video unavailable" :-(
Strange.. try https://youtu.be/r-TLSBdHe1A
Oct 03
prev sibling parent reply =?iso-8859-1?Q?Robert_M._M=FCnch?= <robert.muench saphirion.com> writes:
On 2019-10-03 16:01:19 +0000, Jon Degenhardt said:

 I did a study of LTO and PGO on the combined application and libraries 
 (druntime & phobos). It's available here: 
 https://github.com/eBay/tsv-utils/blob/master/docs/lto-pgo-study.md.
Thanks a lot, very interesting. How did you include druntime & phobos into your app code build environment? What about extended compile times? I'm wondering what would the best dev process look like when doing such a overall LTO+PGO optimization. -- Robert M. MŁnch http://www.saphirion.com smarter | better | faster
Oct 04
parent Jon Degenhardt <jond noreply.com> writes:
On Friday, 4 October 2019 at 08:55:33 UTC, Robert M. M√ľnch wrote:
 On 2019-10-03 16:01:19 +0000, Jon Degenhardt said:

 I did a study of LTO and PGO on the combined application and 
 libraries (druntime & phobos). It's available here: 
 https://github.com/eBay/tsv-utils/blob/master/docs/lto-pgo-study.md.
Thanks a lot, very interesting. How did you include druntime & phobos into your app code build environment? What about extended compile times? I'm wondering what would the best dev process look like when doing such a overall LTO+PGO optimization.
I'm not quite sure what the best build process would look like. For the tsv utilities I put the build steps in make files, but its rather clunky and may not work well for larger projects. I don't have it working with dub. There appears to be a nice opportunity for dub to make the basic cases easier. For the tsv apps the build steps are relatively easy, as individually they are small code bases and have no external dependencies. With the LDC 1.9 release the LTO build part became very easy. The LTO versions of druntime and phobos are shipped with the pre-built binaries, so the LTO part is a couple of compiler switches ('ldc2 -flto=thin -defaultlib=phobos2-ldc-lto,druntime-ldc-lto' see: https://github.com/ldc-developers/ldc/releases/tag/v1.9.0 for details). For PGO I added custom build steps in the makefile, mostly following the steps described in Johan Engelen's blog post: https://johanengelen.github.io/ldc/2016/07/15/Profile-Guided-Optimization-with-LDC.html I did create a very simple app built with LTO & PGO using dub, including external dependencies here: https://github.com/jondegenhardt/dcat-perf. The dub.json file is short enough you'll be able to read it and see the basic steps it takes. Despite that the dub.json file is short, I found it rather tricky to get correct. --Jon
Oct 04