www.digitalmars.com         C & C++   DMDScript  

c++ - Benchmarks

reply "Javier Gutiérrez" <nikkho hotmail.com> writes:
    Are there any benchmarks that compare DMC with other C/C++ compilers?

    In particular I am very interested about the performance of DMC against
BCC, VC++, Watcom C++ and GCC.
Jul 10 2001
next sibling parent reply Jan Knepper <jan smartsoft.cc> writes:
What kind of Benchmarks are you looking for?
Compile time?
Link time?
Executable size?
Execution time of generated executable?



"Javier Gutiérrez" wrote:

     Are there any benchmarks that compare DMC with other C/C++ compilers?

     In particular I am very interested about the performance of DMC against
 BCC, VC++, Watcom C++ and GCC.

Jul 10 2001
parent reply Josh Soref <soref wam.umd.edu> writes:
Jan Knepper wrote:
 What kind of Benchmarks are you looking for?
 Compile time?
 Link time?
 Executable size?
 Execution time of generated executable?

The only benchmark i've found interesting (not useful, not valid, just interesting) was compiling and running the TwoFish encryption algorithm. iirc The authors compared MSVC, Borland, and hand coded assembler. http://www.counterpane.com/twofish.html For things like twofish, you usually want executable size or executable speed to be good (twofish can be tweaked to help out for either of those features). For larger projects like mozilla (yeah i'll try again RSN) compile time, link time and execution performance are probably the most important, lesser would be executable size. <- getting numbers from building mozilla is really too complicated to be practical. Usually you want to do depend builds but build time measurements require clobbers. Execution performance is kind of measurable, but then you have the same sort of questions,.. what and how to measure.
Jul 10 2001
parent reply Jan Knepper <jan smartsoft.cc> writes:
I have written some benchmark test for basically all the things I mentioned.
However, I am not to the point yet were I want to publish the stuff...

My general conclusion so far is that DMC++ usually compiles quite a bit faster
than *any* competitor. I've tried this with small projects, but also with
projects of over 200 source files. (Just compiled a project of 260 source
files...). Surprizingly DMC++ on top taking the shortest time to compile and
build also generated the smallest and actually fastest executables... These two
together are reasons for me to stay with DMC++ dispite the lack of some later
C++ language additions that have not been implemented yet. I would guess that
my benchmarks are limited as I have not yet tested any complicated mathematical
stuff. For that reason I still hesitate to publish my experience also I have
mentioned these results before.

Jan



Josh Soref wrote:

 Jan Knepper wrote:
 What kind of Benchmarks are you looking for?
 Compile time?
 Link time?
 Executable size?
 Execution time of generated executable?

The only benchmark i've found interesting (not useful, not valid, just interesting) was compiling and running the TwoFish encryption algorithm. iirc The authors compared MSVC, Borland, and hand coded assembler. http://www.counterpane.com/twofish.html For things like twofish, you usually want executable size or executable speed to be good (twofish can be tweaked to help out for either of those features). For larger projects like mozilla (yeah i'll try again RSN) compile time, link time and execution performance are probably the most important, lesser would be executable size. <- getting numbers from building mozilla is really too complicated to be practical. Usually you want to do depend builds but build time measurements require clobbers. Execution performance is kind of measurable, but then you have the same sort of questions,.. what and how to measure.

Jul 10 2001
parent reply "Javier Gutiérrez" <nikkho hotmail.com> writes:
I am interested specilly in spped and size of the generated executable.

Could you tell me the numbers, seconds and KBs of your tests? Which are the
other compilers in your tests?


"Jan Knepper" <jan smartsoft.cc> escribió en el mensaje
news:3B4B62EC.137F13E7 smartsoft.cc...
 I have written some benchmark test for basically all the things I

stuff...
 My general conclusion so far is that DMC++ usually compiles quite a bit

with projects of over 200 source files. (Just compiled a project of 260 source files...). Surprizingly DMC++ on top taking the shortest time to compile and build also generated the smallest and actually fastest executables... These two together are reasons for me to stay with DMC++ dispite the lack of some later
 C++ language additions that have not been implemented yet. I would guess

mathematical stuff. For that reason I still hesitate to publish my experience also I have mentioned these results before.
 Jan



 Josh Soref wrote:

 Jan Knepper wrote:
 What kind of Benchmarks are you looking for?
 Compile time?
 Link time?
 Executable size?
 Execution time of generated executable?

The only benchmark i've found interesting (not useful, not valid, just


iirc The authors compared MSVC, Borland, and hand coded assembler. http://www.counterpane.com/twofish.html
 For things like twofish, you usually want executable size or executable


features).
 For larger projects like mozilla (yeah i'll try again RSN) compile time,


would be executable size. <- getting numbers from building mozilla is really too complicated to be practical. Usually you want to do depend builds but build time measurements require clobbers. Execution performance is kind of measurable, but then you have the same sort of questions,.. what and how to measure.

Jul 11 2001
parent Jan Knepper <jan smartsoft.cc> writes:
"Javier Gutiérrez" wrote:

 I am interested specilly in spped and size of the generated executable.

OK, I can get you those.
 Could you tell me the numbers, seconds and KBs of your tests? Which are the
 other compilers in your tests?

Borland C++ Builder V5.0 Microsoft Visual C-- 6.0 I also used to compile with Watcom C++, but I do not want to Install it again... Jan
 "Jan Knepper" <jan smartsoft.cc> escribió en el mensaje
 news:3B4B62EC.137F13E7 smartsoft.cc...
 I have written some benchmark test for basically all the things I

stuff...
 My general conclusion so far is that DMC++ usually compiles quite a bit

with projects of over 200 source files. (Just compiled a project of 260 source files...). Surprizingly DMC++ on top taking the shortest time to compile and build also generated the smallest and actually fastest executables... These two together are reasons for me to stay with DMC++ dispite the lack of some later
 C++ language additions that have not been implemented yet. I would guess

mathematical stuff. For that reason I still hesitate to publish my experience also I have mentioned these results before.
 Jan



 Josh Soref wrote:

 Jan Knepper wrote:
 What kind of Benchmarks are you looking for?
 Compile time?
 Link time?
 Executable size?
 Execution time of generated executable?

The only benchmark i've found interesting (not useful, not valid, just


iirc The authors compared MSVC, Borland, and hand coded assembler. http://www.counterpane.com/twofish.html
 For things like twofish, you usually want executable size or executable


features).
 For larger projects like mozilla (yeah i'll try again RSN) compile time,


would be executable size. <- getting numbers from building mozilla is really too complicated to be practical. Usually you want to do depend builds but build time measurements require clobbers. Execution performance is kind of measurable, but then you have the same sort of questions,.. what and how to measure.


Jul 11 2001
prev sibling parent reply "jacob navia" <jacob jacob.remcomp.fr> writes:
"Javier Gutiérrez" <nikkho hotmail.com> wrote in message
news:9iffn8$1r78$1 digitaldaemon.com...
     Are there any benchmarks that compare DMC with other C/C++ compilers?

     In particular I am very interested about the performance of DMC

 BCC, VC++, Watcom C++ and GCC.

is a good real life application: it makes heavy use of I/O, calculations, etc. Optimizations on in all compilers (dmc flag: -o, lcc flag -O, gcc flag -O9, msvc flag -Ox) To test the generated compilers I used a preprocessed C source (274K) NOTES: MSVC 6.0: Crashes during compilation with: d:\lcc\egcs\reload.c(2852) : fatal error C1001: INTERNAL COMPILER ERROR (compiler file 'f:\vs70builds\9030\vc\Compiler\Utc\src\P2\main.c', line 148) I used MSVC 4.2 instead Borlands free compiler generates a faulty executable (crashes during execution). Compilation time DMC 325.29 seconds lcc-win32: 38.7 seconds gcc: 207.3 seconds MSVC: 64.112 seconds Execution time DMC 6.038 seconds lcc-win32: 6.77 seconds gcc 5.92 seconds MSVC4.2 5.728 seconds Executable size (bytes) DMC 1 890 332 lcc-win32: 1 778 208 gcc: 1 956 864 MSVC 1 884 672 Discussion: 1: The option /IMPLIBC made the linker of DMC crash, so the executable size mentioned above includes the size of the C library that was statically linked. This should reduce the size of the DMC executable by 40K approx. 2: MSVC 6.0 should have approx twice the compilation time of MSVC 4.2 and approx the same speed. A bigger executable size too, since progress is progress :-) 3: To minimize the influence of I/O, I used a source code with a BIG switch statement (approx 600 cases) that makes the gcc code stop for at least 4-5 seconds, almost all the compilation time is spent in optimizing that switch statement, so the influence of I/O is minimized but still present. 4: I used a Windows 2000 system with an Athlon 650MHZ CPU and 256MB of RAM. 5: Without optimization, the speed of DMC is INCREDIBLE (26 seconds!!!). The execution speed increses to 7.6 seconds only, and the executable size increases to 2 115 100. It beats even lcc-win32!
Jul 24 2001
next sibling parent "Walter" <walter digitalmars.com> writes:
Make sure you use -6 on dmc. Some versions used -5 (Pentium scheduling),
which is not optimal for later CPUs. -Walter

jacob navia wrote in message <9jjjvc$16o9$1 digitaldaemon.com>...
"Javier Gutiérrez" <nikkho hotmail.com> wrote in message
news:9iffn8$1r78$1 digitaldaemon.com...
     Are there any benchmarks that compare DMC with other C/C++ compilers?

     In particular I am very interested about the performance of DMC

 BCC, VC++, Watcom C++ and GCC.


is a good real life application: it makes heavy use of I/O, calculations,
etc.

Optimizations on in all compilers (dmc flag: -o, lcc flag -O, gcc flag -O9,
msvc flag -Ox)
To test the generated compilers I used a preprocessed C source (274K)

NOTES:
MSVC 6.0: Crashes during compilation with:
d:\lcc\egcs\reload.c(2852) : fatal error C1001: INTERNAL COMPILER ERROR
                (compiler file
'f:\vs70builds\9030\vc\Compiler\Utc\src\P2\main.c', line 148)
I used MSVC 4.2 instead

Borlands free compiler generates a faulty executable (crashes during
execution).

Compilation time
DMC       325.29 seconds
lcc-win32: 38.7 seconds
gcc:          207.3 seconds
MSVC:     64.112 seconds

Execution time
DMC         6.038 seconds
lcc-win32:  6.77 seconds
gcc             5.92 seconds
MSVC4.2   5.728 seconds

Executable size (bytes)
DMC       1 890 332
lcc-win32: 1 778 208
gcc:           1 956 864
MSVC      1 884 672

Discussion:
1: The option /IMPLIBC made the linker of DMC crash, so the executable size
mentioned above includes the size of the C library that was statically
linked. This should reduce the size of the DMC executable by 40K approx.
2: MSVC 6.0 should have approx twice the compilation time of MSVC 4.2 and
approx the same speed. A bigger executable size too, since progress is
progress :-)
3: To minimize the influence of I/O, I used a source code with a BIG switch
statement (approx 600 cases) that makes the gcc code stop for at least 4-5
seconds, almost all the compilation time is spent in optimizing that switch
statement, so the influence of I/O is minimized but still present.
4: I used a Windows 2000 system with an Athlon 650MHZ CPU and 256MB of RAM.
5: Without optimization, the speed of DMC is INCREDIBLE (26 seconds!!!).

execution speed increses to 7.6 seconds only, and the executable size
increases to 2 115 100. It beats even lcc-win32!

Jul 24 2001
prev sibling parent reply "Javier Gutiérrez" <nikkho hotmail.com> writes:
    That's incredible! Seems that DMC is more stable than BCC 5.5 and MSVC
6!

    I think Walter can improve it with the time in order to generate better
executables...


"jacob navia" <jacob jacob.remcomp.fr> escribió en el mensaje
news:9jjjvc$16o9$1 digitaldaemon.com...
 "Javier Gutiérrez" <nikkho hotmail.com> wrote in message
 news:9iffn8$1r78$1 digitaldaemon.com...
     Are there any benchmarks that compare DMC with other C/C++


     In particular I am very interested about the performance of DMC

 BCC, VC++, Watcom C++ and GCC.


 is a good real life application: it makes heavy use of I/O, calculations,
 etc.

 Optimizations on in all compilers (dmc flag: -o, lcc flag -O, gcc

 msvc flag -Ox)
 To test the generated compilers I used a preprocessed C source (274K)

 NOTES:
 MSVC 6.0: Crashes during compilation with:
 d:\lcc\egcs\reload.c(2852) : fatal error C1001: INTERNAL COMPILER ERROR
                 (compiler file
 'f:\vs70builds\9030\vc\Compiler\Utc\src\P2\main.c', line 148)
 I used MSVC 4.2 instead

 Borlands free compiler generates a faulty executable (crashes during
 execution).

 Compilation time
 DMC       325.29 seconds
 lcc-win32: 38.7 seconds
 gcc:          207.3 seconds
 MSVC:     64.112 seconds

 Execution time
 DMC         6.038 seconds
 lcc-win32:  6.77 seconds
 gcc             5.92 seconds
 MSVC4.2   5.728 seconds

 Executable size (bytes)
 DMC       1 890 332
 lcc-win32: 1 778 208
 gcc:           1 956 864
 MSVC      1 884 672

 Discussion:
 1: The option /IMPLIBC made the linker of DMC crash, so the executable

 mentioned above includes the size of the C library that was statically
 linked. This should reduce the size of the DMC executable by 40K approx.
 2: MSVC 6.0 should have approx twice the compilation time of MSVC 4.2 and
 approx the same speed. A bigger executable size too, since progress is
 progress :-)
 3: To minimize the influence of I/O, I used a source code with a BIG

 statement (approx 600 cases) that makes the gcc code stop for at least 4-5
 seconds, almost all the compilation time is spent in optimizing that

 statement, so the influence of I/O is minimized but still present.
 4: I used a Windows 2000 system with an Athlon 650MHZ CPU and 256MB of

 5: Without optimization, the speed of DMC is INCREDIBLE (26 seconds!!!).

 execution speed increses to 7.6 seconds only, and the executable size
 increases to 2 115 100. It beats even lcc-win32!

Jul 25 2001
parent Jan Knepper <jan smartsoft.cc> writes:
"Javier Gutiérrez" wrote:

     That's incredible! Seems that DMC is more stable than BCC 5.5 and MSVC
 6!

     I think Walter can improve it with the time in order to generate better
 executables...

<g> As if I didn't know.... Jan
Jul 25 2001