www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - std.experimental.randomized_unittest_benchmark is ready for comments

reply Seb <seb wilzba.ch> writes:
Optimizing for performance is one of the major challenges for 
Phobos and user libraries in the next month and years. As an 
example of the benefits, have a look at the recent blog post 
about find [1].

Robert burner Schadek has proposed 
std.experimental.randomized_unittest_benchmark over a year ago. 
This addition tries to tackle two common use cases:

1) Automatically generating test data for algorithms
2) Evaluating the performance of algorithms (aka benchmarking)

The high level idea is to semi-_automatically_ generate 
benchmarks for much more methods than find which allows to check 
for regressions, find outliers and compare the performance to 
other libraries.

As the PR has been pending for so long, it would be quite helpful 
if you could take a look at the proposed addition, test it and 
give your feedback and comments.

Github PR: https://github.com/dlang/phobos/pull/2995
Dub: https://code.dlang.org/packages/std_benchmark

I will handle the review management.

[1] 
https://dlang.org/blog/index.php/2016/06/16/find-was-too-damn-slow-so-we-fixed-it/
Jun 19 2016
parent reply Robert burner Schadek <rburners gmail.com> writes:
Thank you Seb for taking over the review management.

Some additional feature for the proposed module is.

* Simple way to create test data for user defined types
* Benchmark data is stored into csv file for comparing the 
benchmark results between runs
* Standalone tool to create gnuplot graphs from generated csv 
benchmark files

The long time vision (2+ years) is to have a benchmark for every 
function in phobos and have the resulting graph on the Dlang 
webpage (merged PRs -> webpage). So we can track the performance 
of everything.
Jun 19 2016
next sibling parent reply Jack Stouffer <jack jackstouffer.com> writes:
On Sunday, 19 June 2016 at 16:15:15 UTC, Robert burner Schadek 
wrote:

 ...


I would like to try this out on my date parsing library, but I 
don't see a way to generate strings of a specific format.

I'm think you have two options, either pass a std.format format 
string and generate random strings from that, or allow a string 
generating function to be passed.
Jun 19 2016
parent reply Robert burner Schadek <rburners gmail.com> writes:
On Sunday, 19 June 2016 at 18:51:09 UTC, Jack Stouffer wrote:

 I would like to try this out on my date parsing library, but I 
 don't see a way to generate strings of a specific format.


take a look at 
https://github.com/dlang/phobos/pull/2995/files#diff-1a5f159e09980950bb9931ac674cbf40R358

just create a new Gen struct that generates what you want and 
make it a parameter to a function. And you should be done.
Jun 19 2016
parent Jack Stouffer <jack jackstouffer.com> writes:
On Sunday, 19 June 2016 at 19:03:55 UTC, Robert burner Schadek 
wrote:

 On Sunday, 19 June 2016 at 18:51:09 UTC, Jack Stouffer wrote:

 I would like to try this out on my date parsing library, but I 
 don't see a way to generate strings of a specific format.


 take a look at 
 https://github.com/dlang/phobos/pull/2995/files#diff-1a5f159e09980950bb9931ac674cbf40R358

 just create a new Gen struct that generates what you want and 
 make it a parameter to a function. And you should be done.


Hmm, isGen is failing for some reason, and I can't tell why

struct GenDateString
{
     import std.format : formattedWrite;
     import std.random : Random, uniform;
     import std.array : appender;

     string value;

     void gen(ref Random gen)
     {
         auto app = appender!string();

         app.formattedWrite(
             "%s-%s-%s",
             uniform!("[]")(1, 3000),
             uniform!("[]")(1, 12),
             uniform!("[]")(1, 28)
         );

         this.value = app.data;
     }

     ref string opCall()
     {
         return this.value;
     }

     alias opCall this;
}
Jun 29 2016
prev sibling parent Seb <seb wilzba.ch> writes:
On Sunday, 19 June 2016 at 16:15:15 UTC, Robert burner Schadek 
wrote:

 Thank you Seb for taking over the review management.

 Some additional feature for the proposed module is.

 * Simple way to create test data for user defined types
 * Benchmark data is stored into csv file for comparing the 
 benchmark results between runs
 * Standalone tool to create gnuplot graphs from generated csv 
 benchmark files

 The long time vision (2+ years) is to have a benchmark for 
 every function in phobos and have the resulting graph on the 
 Dlang webpage (merged PRs -> webpage). So we can track the 
 performance of everything.


Just a short ping at the general public to give their comments & 
feedback ;-)
Jul 09 2016