www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - DConf 2014 Keynote: High Performance Code Using D by Walter Bright

reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
http://www.reddit.com/r/programming/comments/2aruaf/dconf_2014_keynote_high_performance_code_using_d/

https://www.facebook.com/dlang.org/posts/885322668148082

https://twitter.com/D_Programming/status/489081312297635840


Andrei
Jul 15 2014
next sibling parent reply "John" <john.joyus gmail.com> writes:
On Tuesday, 15 July 2014 at 16:20:34 UTC, Andrei Alexandrescu 
wrote:
 http://www.reddit.com/r/programming/comments/2aruaf/dconf_2014_keynote_high_performance_code_using_d/

 https://www.facebook.com/dlang.org/posts/885322668148082

 https://twitter.com/D_Programming/status/489081312297635840


 Andrei
Thanks for posting these videos. At the end of this video, it sounds like it ends abruptly.. While answering a question, Walter says.. 'it turns out..' and the video ends there.
Jul 15 2014
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/15/2014 11:28 AM, John wrote:
 At the end of this video, it sounds like it ends abruptly..
 While answering a question, Walter says.. 'it turns out..' and the video ends
 there.
That's when my time ran out and I vanished in a puff of greasy black smoke.
Jul 15 2014
parent "John" <john.joyus gmail.com> writes:
On Tuesday, 15 July 2014 at 19:00:35 UTC, Walter Bright wrote:
 On 7/15/2014 11:28 AM, John wrote:
 At the end of this video, it sounds like it ends abruptly..
 While answering a question, Walter says.. 'it turns out..' and 
 the video ends
 there.
That's when my time ran out and I vanished in a puff of greasy black smoke.
:D
Jul 17 2014
prev sibling parent reply Justin Whear <justin economicmodeling.com> writes:
On Tue, 15 Jul 2014 18:28:34 +0000, John wrote:

 On Tuesday, 15 July 2014 at 16:20:34 UTC, Andrei Alexandrescu wrote:
 http://www.reddit.com/r/programming/comments/2aruaf/
dconf_2014_keynote_high_performance_code_using_d/
 https://www.facebook.com/dlang.org/posts/885322668148082

 https://twitter.com/D_Programming/status/489081312297635840


 Andrei
Thanks for posting these videos. At the end of this video, it sounds like it ends abruptly.. While answering a question, Walter says.. 'it turns out..' and the video ends there.
The sentence was "it turns out the simple compiler enhancement I am about to reveal makes all code run 5x faster."
Jul 15 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/15/2014 12:36 PM, Justin Whear wrote:
 The sentence was "it turns out the simple compiler enhancement I am about
 to reveal makes all code run 5x faster."
That does it. You're on the hook for writing my next material!
Jul 15 2014
prev sibling next sibling parent "Joakim" <dlang joakim.airpost.net> writes:
On Tuesday, 15 July 2014 at 16:20:34 UTC, Andrei Alexandrescu 
wrote:
 http://www.reddit.com/r/programming/comments/2aruaf/dconf_2014_keynote_high_performance_code_using_d/

 https://www.facebook.com/dlang.org/posts/885322668148082

 https://twitter.com/D_Programming/status/489081312297635840
Will there be a lower-res video of this talk than 1.3 GBs, as there was for other talks?
Jul 15 2014
prev sibling next sibling parent reply "Jaroslav Hron" <jaroslav.hron gmail.com> writes:
On Tuesday, 15 July 2014 at 16:20:34 UTC, Andrei Alexandrescu 
wrote:
 http://www.reddit.com/r/programming/comments/2aruaf/dconf_2014_keynote_high_performance_code_using_d/

 https://www.facebook.com/dlang.org/posts/885322668148082

 https://twitter.com/D_Programming/status/489081312297635840


 Andrei
Is the presentation itself available somewhere?
Jul 16 2014
parent Nick Sabalausky <SeeWebsiteToContactMe semitwist.com> writes:
On 7/16/2014 5:15 AM, Jaroslav Hron wrote:
 On Tuesday, 15 July 2014 at 16:20:34 UTC, Andrei Alexandrescu wrote:
 http://www.reddit.com/r/programming/comments/2aruaf/dconf_2014_keynote_high_performance_code_using_d/


 https://www.facebook.com/dlang.org/posts/885322668148082

 https://twitter.com/D_Programming/status/489081312297635840


 Andrei
Is the presentation itself available somewhere?
https://archive.org/details/dconf2014-day03-talk01
Jul 17 2014
prev sibling next sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:
 http://www.reddit.com/r/programming/comments/2aruaf/dconf_2014_keynote_high_performance_code_using_d/
Despite Walter is used to "pipeline programming", so the next step is to also handle failures and off-band messages in a functional way (without exceptions and global error values) with two "parallel pipelines", here named "Railway-Oriented Programming". This is one of the simplest introductions (and he can skip the slides 19-53) that I have found of this topic (that in the Haskell community is explained on the base of monads): http://www.slideshare.net/ScottWlaschin/railway-oriented-programming In Bugzilla there are already requests for some Railway-Oriented Programming: https://issues.dlang.org/show_bug.cgi?id=6840 https://issues.dlang.org/show_bug.cgi?id=6843 I think no language extensions are needed for such kind of programming, but of course built-in tuple syntax and basic forms of pattern matching in switch (https://d.puremagic.com/issues/show_bug.cgi?id=596 ) improve the syntax and make the code more handy, handy enough to push more D programmers in using it. For some examples of those things in a system language, this page shows some little examples of functional syntax for Rust: http://science.raphael.poss.name/rust-for-functional-programmers.html Bye, bearophile
Jul 16 2014
next sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
 Despite Walter is
Sorry, I meant to write, "Now Walter is"... Bye, bearophile
Jul 16 2014
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/16/14, 3:22 AM, bearophile wrote:
 Andrei Alexandrescu:
 http://www.reddit.com/r/programming/comments/2aruaf/dconf_2014_keynote_high_performance_code_using_d/
Despite Walter is used to "pipeline programming", so the next step is to also handle failures and off-band messages in a functional way (without exceptions and global error values) with two "parallel pipelines", here named "Railway-Oriented Programming". This is one of the simplest introductions (and he can skip the slides 19-53) that I have found of this topic (that in the Haskell community is explained on the base of monads): http://www.slideshare.net/ScottWlaschin/railway-oriented-programming
Just read the slides, very interesting. I think it would be interesting to experiment with an Expected!T that holds an Algebraic!(T, Exception) as state, and a template like this (warning no slides no understanding, this is very sketchy): template bind(alias fun) { ... } such that given a function e.g. int fun(string a, double b); bind!fun is this function: Expected!int bind!fun(Expected!string a, Expected!double b) { if (a.sux || b.sux) return composeExceptions(a, b); return fun(a.rox, b.rox); } There would also be bindNothrow: Expected!int bindNothrow!fun(Expected!string a, Expected!double b) { if (a.sux || b.sux) return composeExceptions(a, b); try return fun(a.rox, b.rox); catch (Exception e) return e; }
 In Bugzilla there are already requests for some Railway-Oriented
 Programming:

 https://issues.dlang.org/show_bug.cgi?id=6840
Nice, but I think we need Expected!T in addition to Nullable!T.
 https://issues.dlang.org/show_bug.cgi?id=6843
This is not good; trying to see if conversion would succeed is almost as much work as doing it. We need a Expected!To tryTo(From, To)(From source); which produces the error but doesn't throw it.
 I think no language extensions are needed for such kind of programming,
Agreed.
 but of course built-in tuple syntax and basic forms of pattern matching
 in switch (https://d.puremagic.com/issues/show_bug.cgi?id=596 ) improve
 the syntax and make the code more handy, handy enough to push more D
 programmers in using it.
No :o).
 For some examples of those things in a system language, this page shows
 some little examples of functional syntax for Rust:
 http://science.raphael.poss.name/rust-for-functional-programmers.html
We, too, could use a couple of full-time library designers on the roster... Andrei
Jul 19 2014
next sibling parent simendsjo <simendsjo gmail.com> writes:
On 07/19/2014 08:37 PM, Andrei Alexandrescu wrote:
 On 7/16/14, 3:22 AM, bearophile wrote:
 Andrei Alexandrescu:
 http://www.reddit.com/r/programming/comments/2aruaf/dconf_2014_keynote_high_performance_code_using_d/
Despite Walter is used to "pipeline programming", so the next step is to also handle failures and off-band messages in a functional way (without exceptions and global error values) with two "parallel pipelines", here named "Railway-Oriented Programming". This is one of the simplest introductions (and he can skip the slides 19-53) that I have found of this topic (that in the Haskell community is explained on the base of monads): http://www.slideshare.net/ScottWlaschin/railway-oriented-programming
Just read the slides, very interesting.
(...) Didn't look at the slides, but I remember finding the following article a very nice introduction: http://fsharpforfunandprofit.com/posts/recipe-part2/ (...)
Jul 20 2014
prev sibling parent reply "bearophile" <bearophileHUGS lycos.com> writes:
Andrei Alexandrescu:

 Just read the slides, very interesting.
There are many papers, books and articles around that explain the same things, but that explanation is easy to understand even for people not used to functional programming (as I still partially am).
 I think it would be interesting to experiment with an 
 Expected!T that holds an Algebraic!(T, Exception) as state,
In those slides as other member of the sum type they have used an enumeration of possible error conditions (or at first even just strings of the error messages), sometimes augmented with more information, like: | EmailNotValid of EmailAddress | DbAuthorizationError of ConnectionString * Credentials | SmtpTimeout of SmtpConnection | SmtpBadRecipient of EmailAddress
 template bind(alias fun) { ... }

 such that given a function e.g.

 int fun(string a, double b);

 bind!fun is this function:

 Expected!int bind!fun(Expected!string a, Expected!double b) {
   if (a.sux || b.sux) return composeExceptions(a, b);
   return fun(a.rox, b.rox);
 }

 There would also be bindNothrow:

 Expected!int bindNothrow!fun(Expected!string a, Expected!double 
 b) {
   if (a.sux || b.sux) return composeExceptions(a, b);
   try return fun(a.rox, b.rox);
   catch (Exception e) return e;
 }
One of the main points of using those two railways is to avoid exceptions.
 but of course built-in tuple syntax and basic forms of pattern 
 matching
 in switch (https://d.puremagic.com/issues/show_bug.cgi?id=596 
 ) improve
 the syntax and make the code more handy, handy enough to push 
 more D
 programmers in using it.
No :o).
Are you saying you don't want built-in tuples and that you also don't agree with the proposal in issue 596 and that you don't agree that a better syntax doesn't make monads like those actually handy to use? I don't understand what's controversial in what I have written there. From a syntax point of view issue 596 asks for an optional onMatch method, and if you want a syntax to create variables in switch cases. Plus support for structs and classes as variables to switch on. The use of Algebraic, Nullable and Expected is very different (and safer) if you use them through pattern matching. The point is not to ape the functional languages: currently in D Nullable is not much safer (and not more handy) than using a null pointer. Bye, bearophile
Jul 20 2014
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/20/14, 5:57 AM, bearophile wrote:
 In those slides as other member of the sum type they have used an
 enumeration of possible error conditions (or at first even just strings
 of the error messages), sometimes augmented with more information, like:

 | EmailNotValid of EmailAddress
 | DbAuthorizationError of ConnectionString * Credentials
 | SmtpTimeout of SmtpConnection
 | SmtpBadRecipient of EmailAddress
No, those would be stored with the exception (possibly as part of its dynamic type). That's the obvious way to achieve that in D; we don't want to copy verbatim what's most appropriate for other languages.
 template bind(alias fun) { ... }

 such that given a function e.g.

 int fun(string a, double b);

 bind!fun is this function:

 Expected!int bind!fun(Expected!string a, Expected!double b) {
   if (a.sux || b.sux) return composeExceptions(a, b);
   return fun(a.rox, b.rox);
 }

 There would also be bindNothrow:

 Expected!int bindNothrow!fun(Expected!string a, Expected!double b) {
   if (a.sux || b.sux) return composeExceptions(a, b);
   try return fun(a.rox, b.rox);
   catch (Exception e) return e;
 }
One of the main points of using those two railways is to avoid exceptions.
Avoid exceptions as control flow, not as means of passing error information around. I think D's exception chains are very good at the latter. We should use them. Andrei
Jul 20 2014
prev sibling parent reply "Don" <x nospam.com> writes:
On Wednesday, 16 July 2014 at 10:22:41 UTC, bearophile wrote:
 Andrei Alexandrescu:
 http://www.reddit.com/r/programming/comments/2aruaf/dconf_2014_keynote_high_performance_code_using_d/
Despite Walter is used to "pipeline programming", so the next step is to also handle failures and off-band messages in a functional way (without exceptions and global error values) with two "parallel pipelines", here named "Railway-Oriented Programming". This is one of the simplest introductions (and he can skip the slides 19-53) that I have found of this topic (that in the Haskell community is explained on the base of monads): http://www.slideshare.net/ScottWlaschin/railway-oriented-programming In Bugzilla there are already requests for some Railway-Oriented Programming: https://issues.dlang.org/show_bug.cgi?id=6840 https://issues.dlang.org/show_bug.cgi?id=6843 I think no language extensions are needed for such kind of programming, but of course built-in tuple syntax and basic forms of pattern matching in switch (https://d.puremagic.com/issues/show_bug.cgi?id=596 ) improve the syntax and make the code more handy, handy enough to push more D programmers in using it. For some examples of those things in a system language, this page shows some little examples of functional syntax for Rust: http://science.raphael.poss.name/rust-for-functional-programmers.html Bye, bearophile
I think that approach is more convincing for functional languages than for D, especially if you are limited to a single return type. Why not just follow the use Unix stdout/stderr model, and provide an OutputRange for errors to be sent to? I don't really believe that there are two 'railway tracks' in the sense that that presentation implies. Once an error has occurred, typically not much more pipeline processing happens. As for Unix, stdout from one step is tied to stdin, but stderr is output only. There may be further processing of the stderr stream (eg, errors may be reported to a database), but the steps are completely independent from the main stdin-stdout track. I think you get a messy design if you try to combine both into a single pipeline. I think it could be quite interesting to see which algorithms can be created with an Error OutputRange model.
Jul 22 2014
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/22/14, 3:03 AM, Don wrote:
 I don't really believe that there are two 'railway tracks' in the sense
 that that presentation implies. Once an error has occurred, typically
 not much more pipeline processing happens. As for Unix, stdout from one
 step is tied to stdin, but stderr is output only. There may be further
 processing of the stderr stream (eg, errors may be reported to a
 database), but the steps are completely independent from the main
 stdin-stdout track. I think you get a messy design if you try to combine
 both into a single pipeline.
The nice thing is that once the red track is taken, things go through the other functions (which weren't written to take care of errorneous inputs) automatically at a small syntactic cost (the "<>>" operator). -- Andrei
Jul 22 2014
prev sibling next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 7/22/14, 3:03 AM, Don wrote:
 I don't really believe that there are two 'railway tracks' in the sense
 that that presentation implies. Once an error has occurred, typically
 not much more pipeline processing happens. As for Unix, stdout from one
 step is tied to stdin, but stderr is output only. There may be further
 processing of the stderr stream (eg, errors may be reported to a
 database), but the steps are completely independent from the main
 stdin-stdout track. I think you get a messy design if you try to combine
 both into a single pipeline.
The nice thing is that once the red track is taken, things go through the other functions (which weren't written to take care of errorneous inputs) automatically at a small syntactic cost (the ">>" operator). -- Andrei
Jul 22 2014
prev sibling parent "bearophile" <bearophileHUGS lycos.com> writes:
Don:

I think that approach is more convincing for functional 
languages than for D, especially if you are limited to a single 
return type.<
Hopefully someday we'll have good enough tuples in D (including their destructuring), this is similar to having multiple return values. Bye, bearophile
Jul 22 2014
prev sibling next sibling parent reply dennis luehring <dl.soluz gmx.net> writes:
Am 15.07.2014 18:20, schrieb Andrei Alexandrescu:
 http://www.reddit.com/r/programming/comments/2aruaf/dconf_2014_keynote_high_performance_code_using_d/

 https://www.facebook.com/dlang.org/posts/885322668148082

 https://twitter.com/D_Programming/status/489081312297635840


 Andrei
Walter can you give an short (working) example code to show the different resulting assembler for your for-rewrite example - and what compilers your using for testing - only dmd or gdc? this example: T[10] array for(int i = 0; i < 10; ++i) foo(array[i]) i've tested some combination on http://gcc.godbolt.org/ with clang 3.4.1 and gcc4.9x and i can't see any difference
Jul 16 2014
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/16/2014 7:21 AM, dennis luehring wrote:
 can you give an short (working) example code to show the different resulting
 assembler for your for-rewrite example - and what compilers your using for
 testing - only dmd or gdc?
I used dmd.
Jul 17 2014
parent reply dennis luehring <dl.soluz gmx.net> writes:
Am 18.07.2014 04:52, schrieb Walter Bright:
 On 7/16/2014 7:21 AM, dennis luehring wrote:
 can you give an short (working) example code to show the different resulting
 assembler for your for-rewrite example - and what compilers your using for
 testing - only dmd or gdc?
I used dmd.
i sometimes got the feeling that you underestimate the sheer power of todays clang or gcc optimizers - so partly what gdc/ldc can do with your code reminds me of brian schotts exmaple of his sse2 optimized version of his lexer - the dmd generated was much faster then the normal version, but gdc/ldc results of the unoptimized versions are still 50% faster i understand your focus on dmd - but talking about fast code and optimizing WITHOUT even trying to compare with other compiler results is just a little bit strange for someone who stated speed = money
Jul 17 2014
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 7/17/2014 9:40 PM, dennis luehring wrote:
 i understand your focus on dmd - but talking about fast code and optimizing
 WITHOUT even trying to compare with other compiler results is just a little bit
 strange for someone who stated speed = money
The point was to get people to look at the asm output of the compiler, as results can be surprising (as you've also discovered).
Jul 17 2014
parent reply dennis luehring <dl.soluz gmx.net> writes:
Am 18.07.2014 07:54, schrieb Walter Bright:
 On 7/17/2014 9:40 PM, dennis luehring wrote:
 i understand your focus on dmd - but talking about fast code and optimizing
 WITHOUT even trying to compare with other compiler results is just a little bit
 strange for someone who stated speed = money
The point was to get people to look at the asm output of the compiler, as results can be surprising (as you've also discovered).
...of the compilerS - please :) can you post your (full, closed) D array access example from the talk so i don't need to play around with the optimizer to get your asm results
Jul 17 2014
parent Walter Bright <newshound2 digitalmars.com> writes:
On 7/17/2014 11:42 PM, dennis luehring wrote:
 Am 18.07.2014 07:54, schrieb Walter Bright:
 On 7/17/2014 9:40 PM, dennis luehring wrote:
 i understand your focus on dmd - but talking about fast code and optimizing
 WITHOUT even trying to compare with other compiler results is just a little bit
 strange for someone who stated speed = money
The point was to get people to look at the asm output of the compiler, as results can be surprising (as you've also discovered).
...of the compilerS - please :) can you post your (full, closed) D array access example from the talk so i don't need to play around with the optimizer to get your asm results
It's the Warp source code. I don't have a reduced test case.
Jul 18 2014
prev sibling parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Friday, 18 July 2014 at 04:40:52 UTC, dennis luehring wrote:
 Am 18.07.2014 04:52, schrieb Walter Bright:
 On 7/16/2014 7:21 AM, dennis luehring wrote:
 can you give an short (working) example code to show the 
 different resulting
 assembler for your for-rewrite example - and what compilers 
 your using for
 testing - only dmd or gdc?
I used dmd.
i sometimes got the feeling that you underestimate the sheer power of todays clang or gcc optimizers - so partly what gdc/ldc can do with your code reminds me of brian schotts exmaple of his sse2 optimized version of his lexer - the dmd generated was much faster then the normal version, but gdc/ldc results of the unoptimized versions are still 50% faster i understand your focus on dmd - but talking about fast code and optimizing WITHOUT even trying to compare with other compiler results is just a little bit strange for someone who stated speed = money
I think this somewhat misses the point of the example, which I would say was - in academia-speak - purely illustrative. The point still stands, which is "unless you understand the compiler and the architecture, stop trying to second guess performance on the micro-level"
Jul 18 2014
prev sibling parent "Dicebot" <public dicebot.lv> writes:
On Tuesday, 15 July 2014 at 16:20:34 UTC, Andrei Alexandrescu 
wrote:
 http://www.reddit.com/r/programming/comments/2aruaf/dconf_2014_keynote_high_performance_code_using_d/

 https://www.facebook.com/dlang.org/posts/885322668148082

 https://twitter.com/D_Programming/status/489081312297635840


 Andrei
http://youtu.be/eh8WETRT7q4
Jul 16 2014