www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Compiletime Vs Runtime bencmarks

reply "D_Learner" <johnsjdsd gmail.com> writes:
Hello everyone . I need advice on my first D-project . I have 
uploaded it at :-

https://bitbucket.org/mrjohns/matcher/downloads

IDEA : Benchmarking of 3 runtime algorithms and comparing them to 
their compile-time variants. The only difference between these is 
that for the compile time-ones, the lookup tables (i.e. Arrays 
bmBc, bmGs, and suffixes ) must be computed at compile time( I 
currently rely on CTFE ) . While for the runtime-ones the lookup 
tables are computed on runtime.

NB : The pattern matching algorithms themselves need not be 
executed  at compile-time, only the lookup tables.Having stated 
this the algorithms which run on known( compile-time computed) 
tables must be faster than the ones which have to compute them at 
runtime.

My results seem to show something different, only the first   
pair(
BM_Runtime and BM_Compile-time)   yields  admissible results, the 
other two pair give higher execution time for the compile-time 
variants. I think am missing something here. Please help.

Current Results  for the pattern="GCAGAGAG"  are as below :-

BM_Runtime          = 366 hnsecs     position=   513
BM_Compile-time     = 294 hnsecs     position   =513

BMH_Runtime         = 174 hnsecs     position=   513
BMH_Compile-time    = 261 hnsecs     position=   513

AG_Run-time         = 258 hnsecs    position=   513
AG_Compile-time     = 268 hnsecs    position=   513


Running the code : dmd -J.  matcher.d inputs.d  rtime_pre.d 
ctime_pre.d && numactl --physcpubind=0 ./matcher


I would appreciate your suggestions.

Regards,
Aug 17 2015
next sibling parent reply "Edwin van Leeuwen" <edder tkwsping.nl> writes:
On Monday, 17 August 2015 at 14:43:35 UTC, D_Learner wrote:
 Hello everyone . I need advice on my first D-project . I have 
 uploaded it at :-

 Current Results  for the pattern="GCAGAGAG"  are as below :-

 BM_Runtime          = 366 hnsecs     position=   513
 BM_Compile-time     = 294 hnsecs     position   =513

 BMH_Runtime         = 174 hnsecs     position=   513
 BMH_Compile-time    = 261 hnsecs     position=   513

 AG_Run-time         = 258 hnsecs    position=   513
 AG_Compile-time     = 268 hnsecs    position=   513


 Running the code : dmd -J.  matcher.d inputs.d  rtime_pre.d 
 ctime_pre.d && numactl --physcpubind=0 ./matcher
Hi, What happens if you run each algorithm many (say 100000) times. The current times seem to short to be reliable (variation in runtimes would be too great). Regards, Edwin
Aug 17 2015
next sibling parent "D_Learner" <johnsjdsd gmail.com> writes:
On Monday, 17 August 2015 at 14:52:18 UTC, Edwin van Leeuwen 
wrote:
 On Monday, 17 August 2015 at 14:43:35 UTC, D_Learner wrote:
 Hello everyone . I need advice on my first D-project . I have 
 uploaded it at :-

 Current Results  for the pattern="GCAGAGAG"  are as below :-

 BM_Runtime          = 366 hnsecs     position=   513
 BM_Compile-time     = 294 hnsecs     position   =513

 BMH_Runtime         = 174 hnsecs     position=   513
 BMH_Compile-time    = 261 hnsecs     position=   513

 AG_Run-time         = 258 hnsecs    position=   513
 AG_Compile-time     = 268 hnsecs    position=   513


 Running the code : dmd -J.  matcher.d inputs.d  rtime_pre.d 
 ctime_pre.d && numactl --physcpubind=0 ./matcher
Hi, What happens if you run each algorithm many (say 100000) times. The current times seem to short to be reliable (variation in runtimes would be too great). Regards, Edwin
Thyanks, I havent tried that. So it seems you suggest I must fucus on average times. Will try it , the above results are from a single run (no looping), but after executing the program several times I realised that the 1st two seem ok(relatively stable results for same pattern), but the other pair give higher performance for the run-time versions.
Aug 17 2015
prev sibling parent reply "D_Learner" <johnsjdsd gmail.com> writes:
On Monday, 17 August 2015 at 14:52:18 UTC, Edwin van Leeuwen 
wrote:
 On Monday, 17 August 2015 at 14:43:35 UTC, D_Learner wrote:
 Hello everyone . I need advice on my first D-project . I have 
 uploaded it at :-

 Current Results  for the pattern="GCAGAGAG"  are as below :-

 BM_Runtime          = 366 hnsecs     position=   513
 BM_Compile-time     = 294 hnsecs     position   =513

 BMH_Runtime         = 174 hnsecs     position=   513
 BMH_Compile-time    = 261 hnsecs     position=   513

 AG_Run-time         = 258 hnsecs    position=   513
 AG_Compile-time     = 268 hnsecs    position=   513


 Running the code : dmd -J.  matcher.d inputs.d  rtime_pre.d 
 ctime_pre.d && numactl --physcpubind=0 ./matcher
Hi, What happens if you run each algorithm many (say 100000) times. The current times seem to short to be reliable (variation in runtimes would be too great). Regards, Edwin
The surprisingly, the D-profiler gives plausible results:- Algorithm1 2921 int rtime_pre.bm_rmatch (runtime ) 2122 int ctime_pre.bm_cmatch (compiletime ) 1317 int rtime_pre.bmh_rmatch (runtime ) 1099 int ctime_pre.bmh_cmatch (compiletime ) 3959 int rtime_pre.ag_rmatch (runtime ) 2688 pure int ctime_pre.ag_cmatch (compiletime ) This suggests that my timer design has some flaw ;)
Aug 17 2015
parent reply "John Colvin" <john.loughran.colvin gmail.com> writes:
On Monday, 17 August 2015 at 17:48:22 UTC, D_Learner wrote:
 On Monday, 17 August 2015 at 14:52:18 UTC, Edwin van Leeuwen 
 wrote:
 [...]
The surprisingly, the D-profiler gives plausible results:- Algorithm1 2921 int rtime_pre.bm_rmatch (runtime ) 2122 int ctime_pre.bm_cmatch (compiletime ) 1317 int rtime_pre.bmh_rmatch (runtime ) 1099 int ctime_pre.bmh_cmatch (compiletime ) 3959 int rtime_pre.ag_rmatch (runtime ) 2688 pure int ctime_pre.ag_cmatch (compiletime ) This suggests that my timer design has some flaw ;)
std.datetime.StopWatch is the easiest way to do timing manually, or just use std.datetime.benchmark
Aug 17 2015
next sibling parent "D_Learner" <johnsjdsd gmail.com> writes:
On Monday, 17 August 2015 at 22:01:32 UTC, John Colvin wrote:
 On Monday, 17 August 2015 at 17:48:22 UTC, D_Learner wrote:
 On Monday, 17 August 2015 at 14:52:18 UTC, Edwin van Leeuwen 
 wrote:
 [...]
The surprisingly, the D-profiler gives plausible results:- Algorithm1 2921 int rtime_pre.bm_rmatch (runtime ) 2122 int ctime_pre.bm_cmatch (compiletime ) 1317 int rtime_pre.bmh_rmatch (runtime ) 1099 int ctime_pre.bmh_cmatch (compiletime ) 3959 int rtime_pre.ag_rmatch (runtime ) 2688 pure int ctime_pre.ag_cmatch (compiletime ) This suggests that my timer design has some flaw ;)
std.datetime.StopWatch is the easiest way to do timing manually, or just use std.datetime.benchmark
My code already uses std.datetime.StopWatch , but I get results which are no match to the D compilers.
Aug 18 2015
prev sibling parent reply "D_Learner" <johnsjdsd gmail.com> writes:
On Monday, 17 August 2015 at 22:01:32 UTC, John Colvin wrote:
 On Monday, 17 August 2015 at 17:48:22 UTC, D_Learner wrote:
 On Monday, 17 August 2015 at 14:52:18 UTC, Edwin van Leeuwen 
 wrote:
 [...]
The surprisingly, the D-profiler gives plausible results:- Algorithm1 2921 int rtime_pre.bm_rmatch (runtime ) 2122 int ctime_pre.bm_cmatch (compiletime ) 1317 int rtime_pre.bmh_rmatch (runtime ) 1099 int ctime_pre.bmh_cmatch (compiletime ) 3959 int rtime_pre.ag_rmatch (runtime ) 2688 pure int ctime_pre.ag_cmatch (compiletime ) This suggests that my timer design has some flaw ;)
std.datetime.StopWatch is the easiest way to do timing manually, or just use std.datetime.benchmark
My code already uses std.datetime.StopWatch , but I get results which are no match to the D profiler's. Though am not searching for exact, but theyy must be comparable atleast.
Aug 18 2015
parent "John Colvin" <john.loughran.colvin gmail.com> writes:
On Tuesday, 18 August 2015 at 07:16:32 UTC, D_Learner wrote:
 On Monday, 17 August 2015 at 22:01:32 UTC, John Colvin wrote:
 On Monday, 17 August 2015 at 17:48:22 UTC, D_Learner wrote:
 On Monday, 17 August 2015 at 14:52:18 UTC, Edwin van Leeuwen 
 wrote:
 [...]
The surprisingly, the D-profiler gives plausible results:- Algorithm1 2921 int rtime_pre.bm_rmatch (runtime ) 2122 int ctime_pre.bm_cmatch (compiletime ) 1317 int rtime_pre.bmh_rmatch (runtime ) 1099 int ctime_pre.bmh_cmatch (compiletime ) 3959 int rtime_pre.ag_rmatch (runtime ) 2688 pure int ctime_pre.ag_cmatch (compiletime ) This suggests that my timer design has some flaw ;)
std.datetime.StopWatch is the easiest way to do timing manually, or just use std.datetime.benchmark
My code already uses std.datetime.StopWatch , but I get results which are no match to the D profiler's. Though am not searching for exact, but theyy must be comparable atleast.
When you say D profiler, you mean dmd's built-in profiler, yes? That is an instrumenting profiler which adds a not insignificant cost to function calls, which means you have to be careful interpreting the results
Aug 18 2015
prev sibling parent "Atila Neves" <atila.neves gmail.com> writes:
On Monday, 17 August 2015 at 14:43:35 UTC, D_Learner wrote:
 Hello everyone . I need advice on my first D-project . I have 
 uploaded it at :-

 [...]
I wouldn't compare benchmarks without optimisations turned on. Atila
Aug 18 2015