digitalmars.D - Benchmark: OOPack
- Thomas Kuehne (48/48) Nov 04 2004 Another benchmark. OOPack tries to evaluate the cost of using OOP agains...
- no where.com (6/54) Nov 04 2004 This result looks not good for D. I have never seen a benchmark where D
- Thomas Kuehne (15/20) Nov 04 2004 The D port is "quick 'n' dirty hack" because it neither uses static/cons...
- Walter (3/4) Nov 05 2004 The link doesn't work. Can you check it, please?
- =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= (5/12) Nov 05 2004 Maybe Thomas could provide zipped snapshots too ? (http)
- Walter (5/15) Nov 05 2004 [6]
- =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= (5/9) Nov 05 2004 AFAIK, there should be web interfaces to Subversion that
- Thomas Kuehne (4/11) Nov 05 2004 I am going to enable http access soon - before that I have to make sure ...
- Thomas Kuehne (4/15) Nov 06 2004 Http access to DStress and the benchmark sources is up and running.
- Walter (7/10) Nov 06 2004 Thanks. The reason the HAVE_COMPLEX D benchmark is so slow is that class
- no where.com (5/16) Nov 08 2004 Sometime ago, there's a post about "Dropping the distinction between obj...
- Walter (5/20) Nov 09 2004 addressed in
- Ilya Minkov (13/18) Nov 09 2004 The suggestion from this message cannot work without breaking
- Dave (5/25) Nov 05 2004 "finalizing" all of the classes gives much better results for everything...
- Thomas Kuehne (3/14) Nov 05 2004 Could you please send me a diff of your finalized version?
- Dave (24/47) Nov 06 2004 Ok - here that is:
- Dave (7/27) Nov 06 2004 In the NG archives there has been talk about the compiler being able to
- =?ISO-8859-1?Q?Anders_F_Bj=F6rklund?= (36/52) Nov 05 2004 Here are the test results from GDC, running on Mac OS X 10.3:
- Thomas Kuehne (3/10) Nov 05 2004 Thanks, fixed it.
- Thomas Kuehne (42/42) Oct 13 2006 -----BEGIN PGP SIGNED MESSAGE-----
Another benchmark. OOPack tries to evaluate the cost of using OOP against
the cost of the "plain C" style.
The D port is a quick 'n' dirty hack ...
gdc-0.8(gcc-3.4.2): -O2 -frename-registers -fomit-frame-pointer -fweb
-frelease -finline-functions -march=i686
Seconds Mflops
Test Iterations C OOP C OOP Ratio
---- ---------- ----------- ----------- -----
Iterator 100000 0.530 28.990 377.358 6.899 54.698
Complex 1000 0.020 801.01 400.000 0.010 40050.500
Matrix 1000 0.890 7.420 280.899 33.693 8.337
Max 100000 0.860 3.510 116.279 28.490 4.081
dmd-0.105: -O -release -inline
Seconds Mflops
Test Iterations C OOP C OOP Ratio
---- ---------- ----------- ----------- -----
Iterator 100000 1.060 9.910 188.679 20.182 9.349
Complex 1000 0.040 3.990 200.000 2.005 99.750
Matrix 1000 1.460 6.720 171.233 37.202 4.603
Max 100000 0.790 3.530 126.582 28.329 4.468
g++-3.4.2: -O3 -march=i686
Seconds Mflops
Test Iterations C OOP C OOP Ratio
---- ---------- ----------- ----------- -----
Iterator 100000 0.5 0.5 377.4 377.4 1.0
Complex 1000 0.0 0.1 400.0 57.1 7.0
Matrix 1000 0.9 0.7 274.7 342.5 0.8
Max 100000 0.8 0.8 126.6 126.6 1.0
icc-8.0: -O3 -march=i686
Seconds Mflops
Test Iterations C OOP C OOP Ratio
---- ---------- ----------- ----------- -----
Iterator 100000 0.5 0.6 444.4 357.1 1.2
Complex 1000 0.0 0.0 400.0 400.0 1.0
Matrix 1000 0.7 0.7 384.6 384.6 1.0
Max 100000 0.7 0.7 137.0 137.0 1.0
gdc's Iterator and Complex test seem really bad. Both the frontend and the
backend generate better results. This might be an integration problem or an
optimization where the init code was in-lined into the OOP benchmark
function.
Interestingly gcc's Matrix-C function is slower than it's Matrix-OPP
function.
Intel's results are that good, because the bought the company(kai.com) that
wrote this benchmark along with their compiler(KAI C++) and tuned icc for
this benchmark.
OOPack (C++ & D) the original C++ page has vanished
svn://svn.kuehne.cn/dstress/benchmark/oopack
Thomas
Nov 04 2004
This result looks not good for D. I have never seen a benchmark where D
performs so bad like this compared with C++. Is it just because "The D port is
a quick 'n' dirty hack"; or some other reasons.
D is designed to be a simpler language than C++, the object model should be more
efficient, right?
In article <cmcp5c$41g$4 digitaldaemon.com>, Thomas Kuehne says...
Another benchmark. OOPack tries to evaluate the cost of using OOP against
the cost of the "plain C" style.
The D port is a quick 'n' dirty hack ...
gdc-0.8(gcc-3.4.2): -O2 -frename-registers -fomit-frame-pointer -fweb
-frelease -finline-functions -march=i686
Seconds Mflops
Test Iterations C OOP C OOP Ratio
---- ---------- ----------- ----------- -----
Iterator 100000 0.530 28.990 377.358 6.899 54.698
Complex 1000 0.020 801.01 400.000 0.010 40050.500
Matrix 1000 0.890 7.420 280.899 33.693 8.337
Max 100000 0.860 3.510 116.279 28.490 4.081
dmd-0.105: -O -release -inline
Seconds Mflops
Test Iterations C OOP C OOP Ratio
---- ---------- ----------- ----------- -----
Iterator 100000 1.060 9.910 188.679 20.182 9.349
Complex 1000 0.040 3.990 200.000 2.005 99.750
Matrix 1000 1.460 6.720 171.233 37.202 4.603
Max 100000 0.790 3.530 126.582 28.329 4.468
g++-3.4.2: -O3 -march=i686
Seconds Mflops
Test Iterations C OOP C OOP Ratio
---- ---------- ----------- ----------- -----
Iterator 100000 0.5 0.5 377.4 377.4 1.0
Complex 1000 0.0 0.1 400.0 57.1 7.0
Matrix 1000 0.9 0.7 274.7 342.5 0.8
Max 100000 0.8 0.8 126.6 126.6 1.0
icc-8.0: -O3 -march=i686
Seconds Mflops
Test Iterations C OOP C OOP Ratio
---- ---------- ----------- ----------- -----
Iterator 100000 0.5 0.6 444.4 357.1 1.2
Complex 1000 0.0 0.0 400.0 400.0 1.0
Matrix 1000 0.7 0.7 384.6 384.6 1.0
Max 100000 0.7 0.7 137.0 137.0 1.0
gdc's Iterator and Complex test seem really bad. Both the frontend and the
backend generate better results. This might be an integration problem or an
optimization where the init code was in-lined into the OOP benchmark
function.
Interestingly gcc's Matrix-C function is slower than it's Matrix-OPP
function.
Intel's results are that good, because the bought the company(kai.com) that
wrote this benchmark along with their compiler(KAI C++) and tuned icc for
this benchmark.
OOPack (C++ & D) the original C++ page has vanished
svn://svn.kuehne.cn/dstress/benchmark/oopack
Thomas
Nov 04 2004
no where.com schrieb am Donnerstag, 4. November 2004 23:19:This result looks not good for D. I have never seen a benchmark where D performs so bad like this compared with C++. Is it just because "The D port is a quick 'n' dirty hack"; or some other reasons.The D port is "quick 'n' dirty hack" because it neither uses static/const attribute nor D's on constructs like "foreach". Currently it's only in the state where it might uncover serious problems like gdc's "Complex" test case. This benchmark requires review before it can provide comparable data. Comeon every one and tune this monster. ;) svn://svn.kuehne.cn/dstress/benchmark/oopackD is designed to be a simpler language than C++, the object model should be more efficient, right?Guess so too, but keep in mind that - compared to dmc, gcc and icc - the current D compilers are relativly young. To figure out what is going on someone with some time to spare and a dual boot Linux(dmd,gdc,gcc)/Windows(dmc,dmd,gdc,gcc) box should run the benchmark. This way it should be possible to figure if the problem lies within the frontent, backend, benchmark port or the interaction of the tool chain. Thomas
Nov 04 2004
"Thomas Kuehne" <thomas-dloop kuehne.cn> wrote in message news:cmei3a$34n$1 digitaldaemon.com...svn://svn.kuehne.cn/dstress/benchmark/oopackThe link doesn't work. Can you check it, please?
Nov 05 2004
Walter wrote:It seems to work fine from here: (it's a Subversion link)svn://svn.kuehne.cn/dstress/benchmark/oopackThe link doesn't work. Can you check it, please?[6] A oopack/oopack_v1p8_d.d A oopack/oopack_v1p8_cpp.cpp Checked out revision 125.Maybe Thomas could provide zipped snapshots too ? (http) --anders PS. Subversion software at http://subversion.tigris.org/
Nov 05 2004
"Anders F Björklund" <afb algonet.se> wrote in message news:cmfj31$21j4$1 digitaldaemon.com...Walter wrote:[6]It seems to work fine from here: (it's a Subversion link)svn://svn.kuehne.cn/dstress/benchmark/oopackThe link doesn't work. Can you check it, please?Or just make .txt files out of the code and link to them? That way we can examine the code without needing to download, unzip, etc.A oopack/oopack_v1p8_d.d A oopack/oopack_v1p8_cpp.cpp Checked out revision 125.Maybe Thomas could provide zipped snapshots too ? (http)
Nov 05 2004
Walter wrote:AFAIK, there should be web interfaces to Subversion that let's you do both things automatically from the repository... I know such things exists for CVS, at least. --andersMaybe Thomas could provide zipped snapshots too ? (http)Or just make .txt files out of the code and link to them? That way we can examine the code without needing to download, unzip, etc.
Nov 05 2004
Anders F Björklund schrieb:I am going to enable http access soon - before that I have to make sure that the different access rights are correct. ThomasAFAIK, there should be web interfaces to Subversion that let's you do both things automatically from the repository... I know such things exists for CVS, at least.Maybe Thomas could provide zipped snapshots too ? (http)Or just make .txt files out of the code and link to them? That way we can examine the code without needing to download, unzip, etc.
Nov 05 2004
Http access to DStress and the benchmark sources is up and running. http://svn.kuehne.cn/dstress svn://svn.kuehne.cn/dstress ThomasI am going to enable http access soon - before that I have to make sure that the different access rights are correct.AFAIK, there should be web interfaces to Subversion that let's you do both things automatically from the repository... I know such things exists for CVS, at least.Maybe Thomas could provide zipped snapshots too ? (http)Or just make .txt files out of the code and link to them? That way we can examine the code without needing to download, unzip, etc.
Nov 06 2004
"Thomas Kuehne" <thomas-dloop kuehne.cn> wrote in message news:cmjgps$2t9i$1 digitaldaemon.com...Http access to DStress and the benchmark sources is up and running. http://svn.kuehne.cn/dstress svn://svn.kuehne.cn/dstressThanks. The reason the HAVE_COMPLEX D benchmark is so slow is that class Complex, in D, is created on the heap, whereas it is used as strictly a value object. The first solution is to make it a struct instead of a class, which will make it a value object. Even better, just replace class Complex with cdouble, which is D's complex double native type.
Nov 06 2004
In article <cmjtmn$c0f$1 digitaldaemon.com>, Walter says..."Thomas Kuehne" <thomas-dloop kuehne.cn> wrote in message news:cmjgps$2t9i$1 digitaldaemon.com...Sometime ago, there's a post about "Dropping the distinction between objects and references may hinder performance", do you think this issue will be addressed in the future? http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/11960Http access to DStress and the benchmark sources is up and running. http://svn.kuehne.cn/dstress svn://svn.kuehne.cn/dstressThanks. The reason the HAVE_COMPLEX D benchmark is so slow is that class Complex, in D, is created on the heap, whereas it is used as strictly avalue object. The first solution is to make it a struct instead of a class, which will make it a value object. Even better, just replace class Complex with cdouble, which is D's complex double native type.
Nov 08 2004
"Walter" <newshound digitalmars.com> <no where.com> wrote in message news:cmolmi$107v$1 digitaldaemon.com...In article <cmjtmn$c0f$1 digitaldaemon.com>, Walter says...objects and"Thomas Kuehne" <thomas-dloop kuehne.cn> wrote in message news:cmjgps$2t9i$1 digitaldaemon.com...Sometime ago, there's a post about "Dropping the distinction betweenHttp access to DStress and the benchmark sources is up and running. http://svn.kuehne.cn/dstress svn://svn.kuehne.cn/dstressThanks. The reason the HAVE_COMPLEX D benchmark is so slow is that class Complex, in D, is created on the heap, whereas it is used as strictly areferences may hinder performance", do you think this issue will beaddressed inthe future? http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/11960I understand the issue with it, but I want to run with what we've got for a while.
Nov 09 2004
no where.com schrieb:Sometime ago, there's a post about "Dropping the distinction between objects and references may hinder performance", do you think this issue will be addressed in the future? http://www.digitalmars.com/drn-bin/wwwnews?digitalmars.D/11960The suggestion from this message cannot work without breaking polymorphy. Reason: every object can be substituted by an instance of a derived object, which may be larger than the original one. Thus it is unclear how much space should be left within the container to accomodate the object. Thus, it *has* to be allocated dynamically. Suggested solution: allow structs to inherit from classes - the resulting struct is reliable, that is it cannot be substituted by anything else. We attempt to write a complete, detailed proposal, with reasoning and details for suggested implementations, as soon as Walter calls out to design D 2.0. Until then, it is not realistic to expect such features. -eye
Nov 09 2004
In article <cmei3a$34n$1 digitaldaemon.com>, Thomas Kuehne says...no where.com schrieb am Donnerstag, 4. November 2004 23:19:"finalizing" all of the classes gives much better results for everything except Complex - the main problem there is that several new Complex objects are created for each iteration of the oop_test.. Anyone have a solution for that?This result looks not good for D. I have never seen a benchmark where D performs so bad like this compared with C++. Is it just because "The D port is a quick 'n' dirty hack"; or some other reasons.The D port is "quick 'n' dirty hack" because it neither uses static/const attribute nor D's on constructs like "foreach". Currently it's only in the state where it might uncover serious problems like gdc's "Complex" test case. This benchmark requires review before it can provide comparable data. Comeon every one and tune this monster. ;) svn://svn.kuehne.cn/dstress/benchmark/oopackD is designed to be a simpler language than C++, the object model should be more efficient, right?Guess so too, but keep in mind that - compared to dmc, gcc and icc - the current D compilers are relativly young. To figure out what is going on someone with some time to spare and a dual boot Linux(dmd,gdc,gcc)/Windows(dmc,dmd,gdc,gcc) box should run the benchmark. This way it should be possible to figure if the problem lies within the frontent, backend, benchmark port or the interaction of the tool chain. Thomas
Nov 05 2004
Dave schrieb:Could you please send me a diff of your finalized version? ThomasThe D port is "quick 'n' dirty hack" because it neither uses static/const attributes nor D's own constructs like "foreach". Currently it's only in the state where it might uncover serious problems like gdc's "Complex" test case. This benchmark requires review before it can provide comparable data. Come on every one and tune this monster. ;) svn://svn.kuehne.cn/dstress/benchmark/oopack"finalizing" all of the classes gives much better results for everything except Complex - the main problem there is that several new Complex objects are created for each iteration of the oop_test.. Anyone have a solution for that?
Nov 05 2004
In article <cmh819$2ij6$1 digitaldaemon.com>, Thomas Kuehne says...Dave schrieb:Ok - here that is: 322c322 < final class MaxBenchmark : Benchmark { ---Could you please send me a diff of your finalized version? ThomasThe D port is "quick 'n' dirty hack" because it neither uses static/const attributes nor D's own constructs like "foreach". Currently it's only in the state where it might uncover serious problems like gdc's "Complex" test case. This benchmark requires review before it can provide comparable data. Come on every one and tune this monster. ;) svn://svn.kuehne.cn/dstress/benchmark/oopack"finalizing" all of the classes gives much better results for everything except Complex - the main problem there is that several new Complex objects are created for each iteration of the oop_test.. Anyone have a solution for that?class MaxBenchmark : Benchmark {380c380 < final class MatrixBenchmark: public Benchmark { ---class MatrixBenchmark: public Benchmark {405c405 < final class Matrix { ---class Matrix {480c480 < final class IteratorBenchmark: public Benchmark { ---class IteratorBenchmark: public Benchmark {498c498 < final class Iterator { ---class Iterator {566c566 < final class ComplexBenchmark: public Benchmark { ---class ComplexBenchmark: public Benchmark {585c585 < final class Complex { ---class Complex {- Dave
Nov 06 2004
In article <cmei3a$34n$1 digitaldaemon.com>, Thomas Kuehne says...no where.com schrieb am Donnerstag, 4. November 2004 23:19:In the NG archives there has been talk about the compiler being able to 'auto-finalize' classes and methods because of how the D language has been designed. Once that is done a lot of the abstraction penalty should disappear w/o the programmer being concerned with using language constructs to accomplish that. - DaveThis result looks not good for D. I have never seen a benchmark where D performs so bad like this compared with C++. Is it just because "The D port is a quick 'n' dirty hack"; or some other reasons.The D port is "quick 'n' dirty hack" because it neither uses static/const attribute nor D's on constructs like "foreach". Currently it's only in the state where it might uncover serious problems like gdc's "Complex" test case. This benchmark requires review before it can provide comparable data. Comeon every one and tune this monster. ;) svn://svn.kuehne.cn/dstress/benchmark/oopackD is designed to be a simpler language than C++, the object model should be more efficient, right?Guess so too, but keep in mind that - compared to dmc, gcc and icc - the current D compilers are relativly young. To figure out what is going on someone with some time to spare and a dual boot Linux(dmd,gdc,gcc)/Windows(dmc,dmd,gdc,gcc) box should run the benchmark. This way it should be possible to figure if the problem lies within the frontent, backend, benchmark port or the interaction of the tool chain. Thomas
Nov 06 2004
Thomas Kuehne wrote:Another benchmark. OOPack tries to evaluate the cost of using OOP against the cost of the "plain C" style. The D port is a quick 'n' dirty hack ...Here are the test results from GDC, running on Mac OS X 10.3: g++ 3.4.2: (g++ -O3 -mcpu=G4) Seconds Mflops Test Iterations C OOP C OOP Ratio ---- ---------- ----------- ----------- ----- Iterator 100000 0.7 0.8 281.7 266.7 1.1 Complex 1000 0.0 0.2 400.0 42.1 9.5 Matrix 1000 0.9 0.9 280.9 274.7 1.0 Max 100000 1.5 1.9 65.8 52.1 1.3 gdc 0.8: (gdc -O2 -fweb -frelease -finline-functions -mcpu=G4) Seconds Mflops Test Iterations C OOP C OOP Ratio ---- ---------- ----------- ----------- ----- Iterator 0 0.000 0.000 nan nan nan Complex 0 0.000 0.000 nan nan nan Matrix 0 0.000 0.000 nan nan nan Max 0 0.000 0.000 nan nan nan Oops! Hmmm, seems like the clock functions are a bit off: std/c/time.d:version (darwin) { const uint CLOCKS_PER_SEC = 100; const uint CLK_TCK = 100; } else { const uint CLOCKS_PER_SEC = 1000; const uint CLK_TCK = 1000; }And the strtol function has the wrong return type: (64, not 32) oopack_v1p8_d.d:version (darwin) { extern (C) int strtol(char *nptr, char **endptr, int base); } else { extern (C) long strtol(char *nptr, char **endptr, int base); }Seconds Mflops Test Iterations C OOP C OOP Ratio ---- ---------- ----------- ----------- ----- Iterator 100000 0.070 1.279 2857.143 156.372 18.271 Complex 1000 0.002 0.483 4000.000 16.563 241.500 Matrix 1000 0.095 0.753 2631.579 332.005 7.926 Max 100000 0.233 0.375 429.185 266.667 1.609 Still seems off by a factor ten, but it did take way longer... Both were using the same local gcc/gdc build, by the way. (i.e. I used the g++ compiler from the gdc installation) --anders PS. "For a run to be accepted, adjust the number of iterations for each test so that each time reported is greater than
Nov 05 2004
Anders F Björklund schrieb:
And the strtol function has the wrong return type: (64, not 32)
oopack_v1p8_d.d:
>version (darwin) {
>extern (C) int strtol(char *nptr, char **endptr, int base);
>} else {
>extern (C) long strtol(char *nptr, char **endptr, int base);
>}
Thanks, fixed it.
Thomas
Nov 05 2004
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
After roughly 2 years(
http://www.digitalmars.com/pnews/read.php?server=news.digitalmars.com&group=digita
mars.D&artnum=12277 )
I've generated new test results for OOPack:
http://dstress.kuehne.cn/benchmark/oopack/oopack-results.pdf
extract:
dmd-0.169 -O -release -inline
Seconds Mflops
Test Iterations C OOP C OOP Ratio
- ---- ---------- ----------- ----------- -----
Iterator 400000 3.280 4.520 243.902 176.991 1.378
Complex 200000 2.020 158.09 792.079 10.121 78.262
Matrix 3000 3.270 4.560 229.358 164.474 1.394
Max 1000000 3.760 3.990 265.957 250.627 1.061
gdc-0.19(gcc-3.4.6) -c -O3 -frelease -finline-functions -m32
Seconds Mflops
Test Iterations C OOP C OOP Ratio
- ---- ---------- ----------- ----------- -----
Iterator 400000 1.090 4.260 733.945 187.793 3.908
Complex 200000 1.360 241.1 1176.471 6.636 177.279
Matrix 3000 1.140 3.530 657.895 212.465 3.096
Max 1000000 4.100 4.100 243.902 243.902 1.000
g++-4.1.1 -O3 -m32
Seconds Mflops
Test Iterations C OOP C OOP Ratio
- ---- ---------- ----------- ----------- -----
Iterator 400000 1.1 1.1 733.9 740.7 1.0
Complex 200000 1.1 1.1 1428.6 1403.5 1.0
Matrix 3000 1.1 1.1 663.7 663.7 1.0
Max 1000000 3.0 3.1 327.9 326.8 1.0
While the gaps between GCC, GDC and DMD have narrowed, "Complex" still
poses a serious problem for GDC and DMD.
The C version of "Iterator" seems to be surprisingly hard for DMD:
DMD / GDC / GCC = 243.902 / 733.945 / 733.9 Mflops
It would be interesting to see how DM, DMD and GDC compete on a
MSWindows system.
Thomas
-----BEGIN PGP SIGNATURE-----
iD8DBQFFL4X7LK5blCcjpWoRAqNYAJ44mG/ooXg0Wpv4OMpp/wP5jTjDeACeOy2I
ThDvTrHyC2/zdQzBbfTEd0U=
=LXOf
-----END PGP SIGNATURE-----
Oct 13 2006