www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Benchmarking Time Unit

reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
Which way of measuring time in std.datetime should I used if I 
want to benchmark a function by the amount of time spent in the 
current thread, not wall-clock time?
Nov 01 2016
parent reply Jonathan M Davis via Digitalmars-d-learn writes:
On Tuesday, November 01, 2016 16:53:51 Nordlöw via Digitalmars-d-learn 
wrote:

 Which way of measuring time in std.datetime should I used if I
 want to benchmark a function by the amount of time spent in the
 current thread, not wall-clock time?


That's what std.datetime.benchmark is for, though unfortunately, it still
uses core.time.TickDuration rather using core.time.MonoTime and
core.time.Duration.

Alternatively, you can always do something like

immutable before = MonoTime.currTime();
// do stuf...
immutable after = MonoTime.currTime();
Duration timeSpent = after - before;

though that just times how long something took rather than running it over
and over again like benchmark does.

I really need to get replacements for the TickDuration-related stuff in
std.datetime into std.experimental so that we can move towards replacing it
and finally getting rid of TickDuration, but I haven't gotten to it yet.

- Jonathan M Davis
Nov 01 2016
parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 1 November 2016 at 18:39:01 UTC, Jonathan M Davis 
wrote:

 Alternatively, you can always do something like

 immutable before = MonoTime.currTime();
 // do stuf...
 immutable after = MonoTime.currTime();
 Duration timeSpent = after - before;


MonoTime has about 5-10 % fluctuations on my laptop. Is this as 
good as it gets?
Nov 01 2016
next sibling parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 1 November 2016 at 20:19:31 UTC, Nordlöw wrote:

 MonoTime has about 5-10 % fluctuations on my laptop. Is this as 
 good as it gets?


Is

https://dlang.org/phobos/core_time.html#.ClockType.threadCPUTime

what I should use if I'm on Linux?
Nov 01 2016
parent Jonathan M Davis via Digitalmars-d-learn writes:
On Tuesday, November 01, 2016 20:21:32 Nordlöw via Digitalmars-d-learn 
wrote:

 On Tuesday, 1 November 2016 at 20:19:31 UTC, Nordlöw wrote:

 MonoTime has about 5-10 % fluctuations on my laptop. Is this as
 good as it gets?


 Is

 https://dlang.org/phobos/core_time.html#.ClockType.threadCPUTime

 what I should use if I'm on Linux?


The default (ClockType.normal) unless you have a really good reason to do
otherwise. And if you're most worried about having high precision (as is
presumably the case if you're benchmarking), ClockType.normal is the best
you get on Linux. You can use ClockType.precise, but on Linux, it's the same
as ClockType.normal. It's just FreeBSD that has a more precise clock. And
really, ClockType.normal is plenty precise. IIRC, it's usually about a tick
per microsecond on Linux.

- Jonathan M Davis
Nov 01 2016
prev sibling next sibling parent Jonathan M Davis via Digitalmars-d-learn writes:
On Tuesday, November 01, 2016 20:19:31 Nordlöw via Digitalmars-d-learn 
wrote:

 On Tuesday, 1 November 2016 at 18:39:01 UTC, Jonathan M Davis

 wrote:

 Alternatively, you can always do something like

 immutable before = MonoTime.currTime();
 // do stuf...
 immutable after = MonoTime.currTime();
 Duration timeSpent = after - before;


 MonoTime has about 5-10 % fluctuations on my laptop. Is this as
 good as it gets?


How precise it is depends on your system. MonoTime uses the system's
monotonic clock at whatever precision it has. And even if it's perfectly
precise, you're going to see fluctuations in how long it takes your code to
run, since it's not the only thing running on your computer. That's part of
why std.datetime.benchmark runs the function over and over again rather
than only running it only once. That way, you get the average performance
rather than the performance for a single run.

- Jonathan m Davis
Nov 01 2016
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 11/1/16 4:19 PM, Nordlöw wrote:

 On Tuesday, 1 November 2016 at 18:39:01 UTC, Jonathan M Davis wrote:

 Alternatively, you can always do something like

 immutable before = MonoTime.currTime();
 // do stuf...
 immutable after = MonoTime.currTime();
 Duration timeSpent = after - before;


 MonoTime has about 5-10 % fluctuations on my laptop. Is this as good as
 it gets?


This is not clear. What is this a percentage of?

If you are hoping that you run code and the total time taken is going to 
be consistent within a few nanoseconds, then I'm sorry to say it's not 
going to happen. If you mean it's varying by 5-10 seconds, then this is 
a different story, but still could be expected.

MonoTime is going to be very accurate. What isn't true is that on a 
laptop being used with a non-realtime OS, you will have exactly 
consistent results for userspace programs.

As Jonathan mentioned, the way around this is to average several runs of 
the same test.

-Steve
Nov 01 2016
parent reply =?UTF-8?B?Tm9yZGzDtnc=?= <per.nordlow gmail.com> writes:
On Tuesday, 1 November 2016 at 20:54:34 UTC, Steven Schveighoffer 
wrote:

 MonoTime has about 5-10 % fluctuations on my laptop. Is this 
 as good as
 it gets?


 This is not clear. What is this a percentage of?


It percentage of execution time.

15 ms +- 2 ms

That seems doesn't look like what you describe.

Benchmark code here:

https://github.com/nordlow/phobos-next/blob/master/src/benchmarkAppend.d
Nov 01 2016
next sibling parent Stefan Koch <uplink.coder googlemail.com> writes:
On Tuesday, 1 November 2016 at 22:55:36 UTC, Nordlöw wrote:

 On Tuesday, 1 November 2016 at 20:54:34 UTC, Steven 
 Schveighoffer wrote:

 MonoTime has about 5-10 % fluctuations on my laptop. Is this 
 as good as
 it gets?


 This is not clear. What is this a percentage of?


 It percentage of execution time.

 15 ms +- 2 ms

 That seems doesn't look like what you describe.

 Benchmark code here:

 https://github.com/nordlow/phobos-next/blob/master/src/benchmarkAppend.d


I supposed you could not have done it with less templates :)
Since you are hitting gc-code all the time it should be expected 
to have varing times.
Nov 01 2016
prev sibling parent reply Steven Schveighoffer <schveiguy yahoo.com> writes:
On 11/1/16 6:55 PM, Nordlöw wrote:

 On Tuesday, 1 November 2016 at 20:54:34 UTC, Steven Schveighoffer wrote:

 MonoTime has about 5-10 % fluctuations on my laptop. Is this as good as
 it gets?


 This is not clear. What is this a percentage of?


 It percentage of execution time.

 15 ms +- 2 ms


2ms variance is not horrendous or unexpected on a non-realtime system.

-Steve
Nov 03 2016
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 11/3/16 9:03 AM, Steven Schveighoffer wrote:

 On 11/1/16 6:55 PM, Nordlöw wrote:

 On Tuesday, 1 November 2016 at 20:54:34 UTC, Steven Schveighoffer wrote:

 MonoTime has about 5-10 % fluctuations on my laptop. Is this as good as
 it gets?


 This is not clear. What is this a percentage of?


 It percentage of execution time.

 15 ms +- 2 ms


 2ms variance is not horrendous or unexpected on a non-realtime system.


Put another way, this is not a symptom of MonoTime having issues.

-Steve
Nov 03 2016