www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Programming language benchmarks

reply Moritz Warning <moritzwarning web.de> writes:
I think this might be interesting; it's also using D.

http://attractivechaos.wordpress.com/2011/04/25/my-programming-language-
benchmarks-plb/
Apr 28 2011
next sibling parent reply Francisco Almeida <francisco.m.almeida gmail.com> writes:
On 28-04-2011 12:57, Moritz Warning wrote:

 I think this might be interesting; it's also using D.

 http://attractivechaos.wordpress.com/2011/04/25/my-programming-language-
 benchmarks-plb/


Oh dear, the regex results for D are dreadful. And the matmul benchmark 
result isn't too encouraging either. It certainly is satisfying when
Apr 28 2011
next sibling parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/28/11 6:10 AM, Francisco Almeida wrote:

 On 28-04-2011 12:57, Moritz Warning wrote:

 I think this might be interesting; it's also using D.

 http://attractivechaos.wordpress.com/2011/04/25/my-programming-language-
 benchmarks-plb/


 Oh dear, the regex results for D are dreadful. And the matmul benchmark
 result isn't too encouraging either. It certainly is satisfying when



Fortunately we have a GSoC project aimed at improving regex.

Andrei
Apr 28 2011
prev sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
On 28.04.2011 15:10, Francisco Almeida wrote:

 On 28-04-2011 12:57, Moritz Warning wrote:

 I think this might be interesting; it's also using D.

 http://attractivechaos.wordpress.com/2011/04/25/my-programming-language-
 benchmarks-plb/


 Oh dear, the regex results for D are dreadful. And the matmul 
 benchmark result isn't too encouraging either. It certainly is 

 Lua 2 with JIT.


I think the sad truth lies in the shy comment at the bottom of the page:

4) I cannot get theD implementation 
<https://github.com/attractivechaos/plb/blob/master/patmch/patmch_v1.d>working. 
Rhino works for theJavascript program 
<https://github.com/attractivechaos/plb/blob/master/patmch/patmch_v1.js>, but 
run extremely slow.

Having fixed a couple of huge bugs in it, I can't really blame him. Also 
see the version of GDC, just no comment.
And BTW I'm putting together very similar test bench, but I'm obsessed 
with having different regex _engines_ not different _languages_.

-- 
Dmitry Olshansky
Apr 28 2011
prev sibling next sibling parent reply Robert Clipsham <robert octarineparrot.com> writes:
On 28/04/2011 11:57, Moritz Warning wrote:

 I think this might be interesting; it's also using D.

 http://attractivechaos.wordpress.com/2011/04/25/my-programming-language-
 benchmarks-plb/


Looks like this is using an old version of gdc along with D1, and it 
performs abysmally on quite a few of the tests, some of them even 
segfault! We may have to work on this!

-- 
Robert
http://octarineparrot.com/
Apr 28 2011
parent reply Robert Clipsham <robert octarineparrot.com> writes:
On 28/04/2011 12:13, Robert Clipsham wrote:

 On 28/04/2011 11:57, Moritz Warning wrote:

 I think this might be interesting; it's also using D.

 http://attractivechaos.wordpress.com/2011/04/25/my-programming-language-
 benchmarks-plb/


 Looks like this is using an old version of gdc along with D1, and it
 performs abysmally on quite a few of the tests, some of them even
 segfault! We may have to work on this!


I also notice lots of nested for loops in some of the tests, which 
should be using slice operations really. I tried making the change 
myself, seems the necessary ones aren't implemented yet though!

-- 
Robert
http://octarineparrot.com/
Apr 28 2011
parent reply Alexander <aldem+dmars nk7.net> writes:
On 28.04.2011 13:24, Robert Clipsham wrote:


 I also notice lots of nested for loops in some of the tests, which should be
using slice operations really.


  This is the whole point of benchmarking, IMHO - to see how good
compiler/optimizer (and VM, if any) are - for (almost) same code, same
algorithm.

  I recall that once I was very surprised that gcc recognized expression ( ((X)
<< n) | ((X) >> (32 - n)) ) as "rotate left" and used single instruction for
this. I wish DMD would do the same ;)

/Alexander
Apr 28 2011
next sibling parent Robert Clipsham <robert octarineparrot.com> writes:
On 28/04/2011 12:37, Alexander wrote:

 On 28.04.2011 13:24, Robert Clipsham wrote:


 I also notice lots of nested for loops in some of the tests, which should be
using slice operations really.


    This is the whole point of benchmarking, IMHO - to see how good
compiler/optimizer (and VM, if any) are - for (almost) same code, same
algorithm.

    I recall that once I was very surprised that gcc recognized expression (
((X)<<  n) | ((X)>>  (32 - n)) ) as "rotate left" and used single instruction
for this. I wish DMD would do the same ;)

 /Alexander


Regardless, gdc should be roughly inline with gcc for benchmarks, given 
that it uses the same backend. Pretty sure there have been some big 
improvements optimization wise since 0.24.

-- 
Robert
http://octarineparrot.com/
Apr 28 2011
prev sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 4/28/11, Alexander <aldem+dmars nk7.net> wrote:

 On 28.04.2011 13:24, Robert Clipsham wrote:


 I also notice lots of nested for loops in some of the tests, which should
 be using slice operations really.


   This is the whole point of benchmarking, IMHO - to see how good
 compiler/optimizer (and VM, if any) are - for (almost) same code, same
 algorithm.


But then these are not programming >language< benchmarks, they are

compiler< benchmarks.


If you can get more performance out of a language with less code and
simpler syntax, then that language is better performing in my book.
Apr 28 2011
parent reply Alexander <aldem+dmars nk7.net> writes:
On 28.04.2011 17:44, Andrej Mitrovic wrote:


 
 But then these are not programming >language< benchmarks, they are >compiler<
benchmarks.


  "compiler benchmark" is something that should measure compilation speed, for
instance, but "language benchmark" shows how good specific language (+
compiler, of course) is in efficiency of compiled code.


 If you can get more performance out of a language with less code and simpler
syntax, then that language is better performing in my book.


  "language" is a pure language without any libraries (there are dozens of each
for any kind of task, anyway).

  After all, libraries are written in language, so, performance of compiled
code matters. You may even use Perl for matrix multiplication, interfacing it
to external asm library, and it will beat pure C ;)

  But of course, it all depends on application - if you can find something
already implemented and use it - then you are right, but, if you have to write
something on your own (something that doesn't exists), and you *need*
performance, then your
priorities will be shifted, I think ;)

/Alexander
Apr 28 2011
parent reply Daniel Gibson <metalcaedes gmail.com> writes:
Am 28.04.2011 18:10, schrieb Alexander:

 On 28.04.2011 17:44, Andrej Mitrovic wrote:
 

 But then these are not programming >language< benchmarks, they are >compiler<
benchmarks.


 
   "compiler benchmark" is something that should measure compilation speed, for
instance, but "language benchmark" shows how good specific language (+
compiler, of course) is in efficiency of compiled code.
 

 If you can get more performance out of a language with less code and simpler
syntax, then that language is better performing in my book.


 
   "language" is a pure language without any libraries (there are dozens of
each for any kind of task, anyway).
 
   After all, libraries are written in language, so, performance of compiled
code matters. You may even use Perl for matrix multiplication, interfacing it
to external asm library, and it will beat pure C ;)
 
   But of course, it all depends on application - if you can find something
already implemented and use it - then you are right, but, if you have to write
something on your own (something that doesn't exists), and you *need*
performance, then your
 priorities will be shifted, I think ;)
 
 /Alexander


But there's no need for a D compiler to optimize loops that just copy
parts of an array into another array (and similar stuff), because in D
you use slices for that - they're (probably) faster and easier to use.
So IMHO it's fair to use slices where possible.
(And they're a language feature and not just part of the library)
Furthermore this particular benchmark is a "programming language
benchmark" and not a compiler benchmark, so it's fair to use every
feature of the language.

Cheers,
- Daniel
Apr 28 2011
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/28/2011 9:19 AM, Daniel Gibson wrote:

 But there's no need for a D compiler to optimize loops that just copy
 parts of an array into another array (and similar stuff), because in D
 you use slices for that - they're (probably) faster and easier to use.
 So IMHO it's fair to use slices where possible.
 (And they're a language feature and not just part of the library)
 Furthermore this particular benchmark is a "programming language
 benchmark" and not a compiler benchmark, so it's fair to use every
 feature of the language.



For example, a lot of effort is expended in C and Fortran compilers "reverse 
engineering" loops so they can be recompiled and optimized as vector
operations. 
I don't see ever bothering with this in D compilers, as D offers a vector
notation.
Apr 28 2011
parent reply Jens Mueller <jens.k.mueller gmx.de> writes:
Walter Bright wrote:

 On 4/28/2011 9:19 AM, Daniel Gibson wrote:

But there's no need for a D compiler to optimize loops that just copy
parts of an array into another array (and similar stuff), because in D
you use slices for that - they're (probably) faster and easier to use.
So IMHO it's fair to use slices where possible.
(And they're a language feature and not just part of the library)
Furthermore this particular benchmark is a "programming language
benchmark" and not a compiler benchmark, so it's fair to use every
feature of the language.


 
 
 For example, a lot of effort is expended in C and Fortran compilers
 "reverse engineering" loops so they can be recompiled and optimized
 as vector operations. I don't see ever bothering with this in D
 compilers, as D offers a vector notation.


If somebody wants to read the elaborate version of the above:
http://drdobbs.com/blogs/229300270
Walter's Dr. Dobb's articles are very good. I only read the new ones
posted. But I have to read the older ones if I find some time.
BTW
In the book "Coder at Work" Fran Allen takes a quite strong position
against C. She was deeply into Fortran compilers. She says: "We have
seriously regressed, since C developed. C has destroyed our ability to
advance the state of the art in automatic optimization, automatic
parallelization, automatic mapping of a high-level language to the
machine."
When I read this some time ago I found that position quite interesting.
But unfortunately I cannot tell how valid it is. Walter with all of his
experience in developing compilers may give a well informed evaluation
on Allen's statement. I'm just curious.

Jens
Apr 28 2011
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/28/11 12:57 PM, Jens Mueller wrote:

 Walter Bright wrote:

 On 4/28/2011 9:19 AM, Daniel Gibson wrote:

 But there's no need for a D compiler to optimize loops that just copy
 parts of an array into another array (and similar stuff), because in D
 you use slices for that - they're (probably) faster and easier to use.
 So IMHO it's fair to use slices where possible.
 (And they're a language feature and not just part of the library)
 Furthermore this particular benchmark is a "programming language
 benchmark" and not a compiler benchmark, so it's fair to use every
 feature of the language.



 For example, a lot of effort is expended in C and Fortran compilers
 "reverse engineering" loops so they can be recompiled and optimized
 as vector operations. I don't see ever bothering with this in D
 compilers, as D offers a vector notation.


 If somebody wants to read the elaborate version of the above:
 http://drdobbs.com/blogs/229300270
 Walter's Dr. Dobb's articles are very good. I only read the new ones
 posted. But I have to read the older ones if I find some time.
 BTW
 In the book "Coder at Work" Fran Allen takes a quite strong position
 against C. She was deeply into Fortran compilers. She says: "We have
 seriously regressed, since C developed. C has destroyed our ability to
 advance the state of the art in automatic optimization, automatic
 parallelization, automatic mapping of a high-level language to the
 machine."
 When I read this some time ago I found that position quite interesting.
 But unfortunately I cannot tell how valid it is. Walter with all of his
 experience in developing compilers may give a well informed evaluation
 on Allen's statement. I'm just curious.

 Jens


It's simple - Fortran can assume that all data is unaliased (although 
aliasing is still technically possible). C, with its intensive 
pointer-based ethos that makes many aliasing-related optimizations 
difficult, has annoyed many a Fortran aficionados.


Andrei
Apr 28 2011
next sibling parent reply Jens Mueller <jens.k.mueller gmx.de> writes:
Andrei Alexandrescu wrote:

 On 4/28/11 12:57 PM, Jens Mueller wrote:

Walter Bright wrote:

On 4/28/2011 9:19 AM, Daniel Gibson wrote:

But there's no need for a D compiler to optimize loops that just copy
parts of an array into another array (and similar stuff), because in D
you use slices for that - they're (probably) faster and easier to use.
So IMHO it's fair to use slices where possible.
(And they're a language feature and not just part of the library)
Furthermore this particular benchmark is a "programming language
benchmark" and not a compiler benchmark, so it's fair to use every
feature of the language.



For example, a lot of effort is expended in C and Fortran compilers
"reverse engineering" loops so they can be recompiled and optimized
as vector operations. I don't see ever bothering with this in D
compilers, as D offers a vector notation.


If somebody wants to read the elaborate version of the above:
http://drdobbs.com/blogs/229300270
Walter's Dr. Dobb's articles are very good. I only read the new ones
posted. But I have to read the older ones if I find some time.
BTW
In the book "Coder at Work" Fran Allen takes a quite strong position
against C. She was deeply into Fortran compilers. She says: "We have
seriously regressed, since C developed. C has destroyed our ability to
advance the state of the art in automatic optimization, automatic
parallelization, automatic mapping of a high-level language to the
machine."
When I read this some time ago I found that position quite interesting.
But unfortunately I cannot tell how valid it is. Walter with all of his
experience in developing compilers may give a well informed evaluation
on Allen's statement. I'm just curious.

Jens


 
 It's simple - Fortran can assume that all data is unaliased
 (although aliasing is still technically possible). C, with its
 intensive pointer-based ethos that makes many aliasing-related
 optimizations difficult, has annoyed many a Fortran aficionados.


Thanks. So if a compiler can assume that pointers do not alias it can
generate much better code. What's D's standpoint on that matter then?
C99 has restrict. I never came across something similar in D.

Jens
Apr 28 2011
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/28/2011 11:44 AM, Jens Mueller wrote:

 Thanks. So if a compiler can assume that pointers do not alias it can
 generate much better code. What's D's standpoint on that matter then?
 C99 has restrict. I never came across something similar in D.


Const and immutable.
Apr 28 2011
next sibling parent reply Jens Mueller <jens.k.mueller gmx.de> writes:
Walter Bright wrote:

 On 4/28/2011 11:44 AM, Jens Mueller wrote:

Thanks. So if a compiler can assume that pointers do not alias it can
generate much better code. What's D's standpoint on that matter then?
C99 has restrict. I never came across something similar in D.


 
 Const and immutable.
 


That allows specifying that the memory pointed to is only readable which
enables doing the optimizations. But what if the memory is writable but
I can my sure that the pointers pointing to it will never alias? How do
I pass that information to the compiler?

Jens
Apr 28 2011
next sibling parent reply Timon Gehr <timon.gehr gmx.ch> writes:
 That allows specifying that the memory pointed to is only readable which
 enables doing the optimizations. But what if the memory is writable but
 I can my sure that the pointers pointing to it will never alias? How do
 I pass that information to the compiler?

 Jens


assert(p1 != p2); (or assert(c1 !is c2) for references).

I do not think DMD takes advantage of such constructs yet though.
Apr 28 2011
parent Jens Mueller <jens.k.mueller gmx.de> writes:
Timon Gehr wrote:

 That allows specifying that the memory pointed to is only readable which
 enables doing the optimizations. But what if the memory is writable but
 I can my sure that the pointers pointing to it will never alias? How do
 I pass that information to the compiler?

 Jens


 
 assert(p1 != p2); (or assert(c1 !is c2) for references).
 
 I do not think DMD takes advantage of such constructs yet though.


Just because the pointers aren't equal doesn't mean that they won't
alias. So I don't see how the above can help. As far as I understand it
the compiler has to know at compile time that the given pointers won't
alias.

Jens
Apr 28 2011
prev sibling next sibling parent bearophile <bearophileHUGS lycos.com> writes:
Jens Mueller:


 That allows specifying that the memory pointed to is only readable which
 enables doing the optimizations. But what if the memory is writable but
 I can my sure that the pointers pointing to it will never alias? How do
 I pass that information to the compiler?


Here's my answer. I think you can't do it, currently. To do it you have to rely
on the programmer like C does, or enforce it statically, like some languages
with advanced type systems do. By general design decision D generally wants to
enforce its attributes, this removes the first option. I presume the second
option requires type system complexities that D doesn't want, at the moment
(like linear types).

Bye,
bearophile
Apr 28 2011
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/28/11 4:14 PM, Jens Mueller wrote:

 Walter Bright wrote:

 On 4/28/2011 11:44 AM, Jens Mueller wrote:

 Thanks. So if a compiler can assume that pointers do not alias it can
 generate much better code. What's D's standpoint on that matter then?
 C99 has restrict. I never came across something similar in D.


 Const and immutable.


 That allows specifying that the memory pointed to is only readable which
 enables doing the optimizations. But what if the memory is writable but
 I can my sure that the pointers pointing to it will never alias? How do
 I pass that information to the compiler?

 Jens


There is no way. I don't know of a language where that is possible in a 
systematic and provable way. C's restricts sets a poor example.

Andrei
Apr 28 2011
parent reply bearophile <bearophileHUGS lycos.com> writes:
Andrei:


 C's restricts sets a poor example.


There are infinite things that a compiler can't prove about the code. Example:
the type system can't prove that  system code that uses pointer arithmetic
contains no bugs, pointers that go past arrays, etc. C99 restrict is just
another example where the language doesn't require a correctness proof, and
relies on the programmer. Allowing a " restrict" attribute in D2  system code
is an option.

Bye,
bearophile
Apr 28 2011
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/28/11 7:53 PM, bearophile wrote:

 Andrei:


 C's restricts sets a poor example.


 There are infinite things that a compiler can't prove about the code.


Restrict is particularly bad. See 
http://www.lysator.liu.se/c/dmr-on-noalias.html (noalias was the initial 
name of restrict).

Andrei
Apr 28 2011
prev sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Walter:


 On 4/28/2011 11:44 AM, Jens Mueller wrote:

 Thanks. So if a compiler can assume that pointers do not alias it can
 generate much better code. What's D's standpoint on that matter then?
 C99 has restrict. I never came across something similar in D.


 
 Const and immutable.


I don't understand. "restrict" means "this pointer has no alias". So how is
const/immutable helping foo being optimized assuming a.ptr and b.ptr are
distinct?

void foo(const float[] a, const float b[]) {}
void main() {
  const float[] a = [1.5, 2.5];
  foo(a, a);
}

Bye,
bearophile
Apr 28 2011
next sibling parent bearophile <bearophileHUGS lycos.com> writes:
 "restrict" means "this pointer has no alias".


Jens Mueller is right, it's not just pointers, a.ptr and b.ptr are distinct but
they refer to memory zones that are not distinct:

auto a = [1, 2, 3, 4, 5];
auto b = [1 .. 3];

Bye,
bearophile
Apr 28 2011
prev sibling parent reply Moritz Warning <moritzwarning web.de> writes:
On Thu, 28 Apr 2011 17:44:35 -0400, bearophile wrote:


 Walter:
 

 On 4/28/2011 11:44 AM, Jens Mueller wrote:

 Thanks. So if a compiler can assume that pointers do not alias it can
 generate much better code. What's D's standpoint on that matter then?
 C99 has restrict. I never came across something similar in D.


 
 Const and immutable.


 
 I don't understand. "restrict" means "this pointer has no alias". So how
 is const/immutable helping foo being optimized assuming a.ptr and b.ptr
 are distinct?
 
 void foo(const float[] a, const float b[]) {} void main() {
   const float[] a = [1.5, 2.5];
   foo(a, a);
 }


When the contents are both const, then there won't be any writing
to it, of course. In that case the same optimizations can be applied as 
when they point to different memory.

both arrays point to different memory.
Apr 28 2011
parent bearophile <bearophileHUGS lycos.com> writes:
Moritz Warning:


 When the contents are both const, then there won't be any writing
 to it, of course. In that case the same optimizations can be applied as 
 when they point to different memory.
 
 both arrays point to different memory.


I see, thank you.

Bye,
bearophile
Apr 28 2011
prev sibling parent reply Don <nospam nospam.com> writes:
Jens Mueller wrote:

 Andrei Alexandrescu wrote:

 On 4/28/11 12:57 PM, Jens Mueller wrote:

 Walter Bright wrote:

 On 4/28/2011 9:19 AM, Daniel Gibson wrote:

 But there's no need for a D compiler to optimize loops that just copy
 parts of an array into another array (and similar stuff), because in D
 you use slices for that - they're (probably) faster and easier to use.
 So IMHO it's fair to use slices where possible.
 (And they're a language feature and not just part of the library)
 Furthermore this particular benchmark is a "programming language
 benchmark" and not a compiler benchmark, so it's fair to use every
 feature of the language.


 For example, a lot of effort is expended in C and Fortran compilers
 "reverse engineering" loops so they can be recompiled and optimized
 as vector operations. I don't see ever bothering with this in D
 compilers, as D offers a vector notation.


 If somebody wants to read the elaborate version of the above:
 http://drdobbs.com/blogs/229300270
 Walter's Dr. Dobb's articles are very good. I only read the new ones
 posted. But I have to read the older ones if I find some time.
 BTW
 In the book "Coder at Work" Fran Allen takes a quite strong position
 against C. She was deeply into Fortran compilers. She says: "We have
 seriously regressed, since C developed. C has destroyed our ability to
 advance the state of the art in automatic optimization, automatic
 parallelization, automatic mapping of a high-level language to the
 machine."
 When I read this some time ago I found that position quite interesting.
 But unfortunately I cannot tell how valid it is. Walter with all of his
 experience in developing compilers may give a well informed evaluation
 on Allen's statement. I'm just curious.

 Jens


 It's simple - Fortran can assume that all data is unaliased
 (although aliasing is still technically possible). C, with its
 intensive pointer-based ethos that makes many aliasing-related
 optimizations difficult, has annoyed many a Fortran aficionados.


 
 Thanks. So if a compiler can assume that pointers do not alias it can
 generate much better code. What's D's standpoint on that matter then?
 C99 has restrict. I never came across something similar in D.
 
 Jens


D reduces the use of pointers. You can do much more with arrays.
Apr 28 2011
parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/28/2011 9:10 PM, Don wrote:

 D reduces the use of pointers. You can do much more with arrays.


True, but dynamic arrays offer no inherent protection against overlap, though 
you can add a runtime check for that.
Apr 28 2011
parent Francisco Almeida <francisco.m.almeida gmail.com> writes:
== Quote from Walter Bright (newshound2 digitalmars.com)'s article

 True, but dynamic arrays offer no inherent protection against overlap, though
 you can add a runtime check for that.


Another reason why we should have memory pool support in Phobos.
Apr 28 2011
prev sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 4/28/2011 11:29 AM, Andrei Alexandrescu wrote:

 It's simple - Fortran can assume that all data is unaliased (although aliasing
 is still technically possible). C, with its intensive pointer-based ethos that
 makes many aliasing-related optimizations difficult, has annoyed many a Fortran
 aficionados.


This also segues back into C's biggest mistake:

     http://drdobbs.com/blogs/architecture-and-design/228701625
Apr 28 2011
parent Jens Mueller <jens.k.mueller gmx.de> writes:
Walter Bright wrote:

 On 4/28/2011 11:29 AM, Andrei Alexandrescu wrote:

It's simple - Fortran can assume that all data is unaliased (although aliasing
is still technically possible). C, with its intensive pointer-based ethos that
makes many aliasing-related optimizations difficult, has annoyed many a Fortran
aficionados.


 
 This also segues back into C's biggest mistake:
 
     http://drdobbs.com/blogs/architecture-and-design/228701625


What a coincidence. Just finished template constraints and C's biggest
mistakes is right after that on my list. Also found out that Andrew
Koenig has some nice articles.

Jens
Apr 28 2011
prev sibling parent reply Alexander <aldem+dmars nk7.net> writes:
On 28.04.2011 18:19, Daniel Gibson wrote:


 Furthermore this particular benchmark is a "programming language benchmark"
and not a compiler benchmark, so it's fair to use every feature of the language.


  But applicability of language features is a bit limited, I would say. I
think, this is not exactly fair - to take any domain-specific task (like
"matrix multiplication") and use it as a "language benchmark".

  In this particular case (matrix multiplication) - yes, we may gain from
specific features. But another example - I need to write MIME headers parses in
D - this involves (mostly) scanning of strings, comparisons etc - it is hardly
possible to use
some feature which could really help and will be fast (std.string is mostly
slow as it is a bit too generic, regexps are even more slow).

  So, I've to write my own version - using only core features of the language
(as there is no highly-optimized nor any version in standard library anyway).

  Now, when this is done - what it will be - "compiler benchmark" or "language
benchmark"? To me, of course it does matter - how good is generated code, as it
directly affect performance of my application.

/Alexander
Apr 29 2011
parent reply "Paulo Pinto" <pjmlp progtools.org> writes:
It is always about  compiler benchmark.

There are no slow or fast languages, there are slow or fast implementations 
of a given language.

The only point that you can make about language related performance, is to 
say that a given language
feature cannot be made to run any faster given the best compiler 
implementation for the language.

Having said this, the point is always about implementations.

--
Paulo

"Alexander" <aldem+dmars nk7.net> wrote in message 
news:ipe1d9$1e0u$1 digitalmars.com...

 On 28.04.2011 18:19, Daniel Gibson wrote:


 Furthermore this particular benchmark is a "programming language 
 benchmark" and not a compiler benchmark, so it's fair to use every 
 feature of the language.


  But applicability of language features is a bit limited, I would say. I 
 think, this is not exactly fair - to take any domain-specific task (like 
 "matrix multiplication") and use it as a "language benchmark".

  In this particular case (matrix multiplication) - yes, we may gain from 
 specific features. But another example - I need to write MIME headers 
 parses in D - this involves (mostly) scanning of strings, comparisons 
 etc - it is hardly possible to use
 some feature which could really help and will be fast (std.string is 
 mostly slow as it is a bit too generic, regexps are even more slow).

  So, I've to write my own version - using only core features of the 
 language (as there is no highly-optimized nor any version in standard 
 library anyway).

  Now, when this is done - what it will be - "compiler benchmark" or 
 "language benchmark"? To me, of course it does matter - how good is 
 generated code, as it directly affect performance of my application.

 /Alexander
Apr 29 2011
parent Alexander <aldem+dmars nk7.net> writes:
On 29.04.2011 14:48, Paulo Pinto wrote:


 There are no slow or fast languages, there are slow or fast implementations of
a given language.


  But is is quite seldom that some language has several implementations, that's
why compiler and language benchmarks are bound usually.

  For example - perl - there is only one implementation, and it is slow for
many things.

/Alexander
Apr 29 2011
prev sibling parent reply bearophile <bearophileHUGS lycos.com> writes:
Moritz Warning:


 I think this might be interesting; it's also using D.


I was about to post this :-)
The little benchmark:
http://attractivechaos.wordpress.com/2011/04/25/my-programming-language-benchmarks-plb/
http://attractivechaos.github.com/plb/

(The matrix mul code is not a realistic example of code because people use
library code to do this operation.)

Original C code:

double **mm_mul(int n, double *const *a, double *const *b)
{
    int i, j, k;
    double **m, **c;
    m = mm_init(n); c = mm_init(n);
    for (i = 0; i < n; ++i) // transpose
        for (j = 0; j < n; ++j)
            c[i][j] = b[j][i];
    for (i = 0; i < n; ++i) {
        double *p = a[i], *q = m[i];
        for (j = 0; j < n; ++j) {
            double t = 0.0, *r = c[j];
            for (k = 0; k < n; ++k)
                t += p[k] * r[k];
            q[j] = t;
        }
    }
    mm_destroy(n, c);
    return m;
}


The original D1 code of the matrix mul, coming from the Java version:

double[][] matmul(double[][] a, double[][] b) {
    int m = a.length, n = a[0].length, p = b[0].length;
    double[][] x, c;
    x.length = m; c.length = p;
    for (int i = 0; i < m; ++i) x[i].length = p;
    for (int i = 0; i < p; ++i) c[i].length = n;
    for (int i = 0; i < n; ++i) // transpose
        for (int j = 0; j < p; ++j)
            c[j][i] = b[i][j];
    for (int i = 0; i < m; ++i)
        for (int j = 0; j < p; ++j) {
            double s = 0.0;
            for (int k = 0; k < n; ++k)
                s += a[i][k] * c[j][k];
            x[i][j] = s;
        }
    return x;
}



Improved D2 code:
http://codepad.org/aifQYhjI


pure nothrow double[][] matMul(in double[][] a, in double[][] b) {
    immutable int m = a.length,
                  n = a[0].length,
                  p = b[0].length;

    // transpose
    auto c = new double[][](p, n);
    foreach (i, brow; b)
        foreach (j, bx; brow)
            c[j][i] = bx;

    auto x = new double[][](m, p);

    foreach (i, arow; a)
        foreach (j, crow; c) {
            // x[i][j] = std.numeric.dotProduct(arow, crow); // right way
            double s = 0.0;
            foreach (k, arowk; arow)
                s += arowk * crow[k];
            x[i][j] = s;
        }

    return x;
}



Lua code:

function matrix.mul(a, b, n)
	local y = matrix.new(n, n)
	local c = matrix.T(b, n)
	for i = 1, n - 1 do
		for j = 1, n - 1 do
			local sum = 0
			for k = 0, n - 1 do sum = sum + a[i][k] * c[j][k] end
			y[i][j] = sum
		end
	end
	return y
end

--------------------

Timings, seconds:
  ICC-12.0.3               C    1.8
  GCC-4.3.2                C    2.3
  GDC-0.24                 D    8.8 (DMD gives similar timings)
  Java JRE-1.6.0_14        Java 2.9
  LuaJIT-2.0.0-beta6 (JIT) Lua  2.7

Improved D version: about 4.4 seconds


Inner loop GCC:

L25:
	fldl	(%edx,%eax,8)
	fmull	(%ecx,%eax,8)
	addl	$1, %eax
	cmpl	%ebx, %eax
	faddp	%st, %st(1)
	jne	L25


Inner loop DMD, original version:

L153:	mov	EDX,4[EBX]
		mov	EAX,[EBX]
		mov	EAX,024h[ESP]
		fld	qword ptr [ESI*8][EDX]
		mov	EDX,028h[ESP]
		mov	EAX,[EDI*8][EDX]
		mov	EDX,4[EDI*8][EDX]
		fmul	qword ptr [ESI*8][EDX]
		inc	ESI
		cmp	ESI,EBP
		fadd	qword ptr 034h[ESP]
		fstp	qword ptr 034h[ESP]
		jl	L153


Inner loop DMD, improved:

L172:	fld	qword ptr [EBX*8][ECX]
		fmul	qword ptr [EBX*8][ESI]
		inc	EBX
		cmp	EBX,04Ch[ESP]
		fadd	qword ptr 054h[ESP]
		fstp	qword ptr 054h[ESP]
		jb	L172


The asm produced by DMD perfoms too many memory accesses in the inner loop. Is
it possible to improve the DMD backend to remove this problem?

Bye,
bearophile
Apr 28 2011
next sibling parent Moritz Warning <moritzwarning web.de> writes:
On Thu, 28 Apr 2011 07:50:19 -0400, bearophile wrote:


 Moritz Warning:
 

 I think this might be interesting; it's also using D.


 
 I was about to post this :-)


I was surprised you haven't had already.
Looks like I got lucky. :-)

Have you seen somewhere what compiler options he did use for gdc?
Anyway, I would hope the benchmark would use a more recent version of gdc.
Apr 28 2011
prev sibling next sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 4/28/11 6:50 AM, bearophile wrote:

 Moritz Warning:


 I think this might be interesting; it's also using D.


 I was about to post this :-)
 The little benchmark:
 http://attractivechaos.wordpress.com/2011/04/25/my-programming-language-benchmarks-plb/
 http://attractivechaos.github.com/plb/

 (The matrix mul code is not a realistic example of code because people use
library code to do this operation.)


[snip]

Cristi's GSoC project may target linear algebra operations.

Andrei
Apr 28 2011
parent reply bearophile <bearophileHUGS lycos.com> writes:
Andrei:


 Cristi's GSoC project may target linear algebra operations.


That benchmark code is not a realistic example because don't write matrix
multiplications manually like that. But the benchmark has found a weakness in
array management that's important anyway.

Bye,
bearophile
Apr 28 2011
parent Don <nospam nospam.com> writes:
bearophile wrote:

 Andrei:
 

 Cristi's GSoC project may target linear algebra operations.


 
 That benchmark code is not a realistic example because don't write matrix
multiplications manually like that. But the benchmark has found a weakness in
array management that's important anyway.
 
 Bye,
 bearophile


It's very well known. DMD's floating point code is poor because it NEVER 
stores floating-point loop variables in registers. Ever.
Disassembling almost any piece of DMD floating point code will show 
that. The floating-point backend is about the simplest you could 
imagine. (By contrast, the integer backend is extremely sophisticated).
Apr 28 2011
prev sibling next sibling parent Alexey Prokhin <alexey.prokhin yandex.ru> writes:
I also did a little benchmarks. Here is the results:

Time for multiplying two 1000x1000 matrics:
clang 2.8 (-O3)	                               2.4s
gcc 4.4.5 (-O3)	                               2.5s
ldc2 llvm-2.9 (-O3 -release)	       2.7s
gdc2 4.4.6 (-O3  -frelease -finline)    2.9s
dmd 2.052 (-O -release -inline)	       4.9s

1500x1500:
Apr 29 2011
prev sibling parent Alexey Prokhin <alexey.prokhin yandex.ru> writes:
1500x1500:
Apr 29 2011