www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Weird performance degradation when using vector ops

reply Andrej Mitrovic <none none.none> writes:
I was testing out array concatenation and reassignment performance when I ran
into some weird performance degradation when using vector ops.

I have two dchar[][] buffers, one is initialized via vector ops, the other via
a foreach loop. Both are initialized to the same values, and they compare equal
via the assert.

After that, I compare concatenation and reassignment of both buffers. The
buffer which was initialized via vector ops operates much slower for some
reason.

On my machine, the following code gives these average results:
buffer1 done in usecs: 19820.
buffer2 done in usecs: 650.

Download: https://gist.github.com/965883
Pasted here:

import std.stdio;
import std.algorithm;
import std.array;
import std.datetime;

void testPerf(ref dchar[][] buffer)
{
    buffer = buffer[0..buffer.length/2] ~ "test"d.dup ~ buffer[buffer.length/2
.. $];
}

void main()
{
    dchar[][] buffer1;
    dchar[][] buffer2;
    buffer1.length = 100_000;
    buffer2.length = 100_000;
    
    buffer1[] = "foo"d.replicate(100).dup;
    
    foreach (ref buf; buffer2)
    {
        buf = "foo"d.replicate(100).dup;
    }
    
    assert(buffer1 == buffer2);    // will take a few seconds..
    
    StopWatch sw;
    sw.start();
    testPerf(buffer1);
    sw.stop();
    writefln("buffer1 done in usecs: %s.", sw.peek().usecs);
    
    sw.reset();
    sw.start();
    testPerf(buffer2);
    sw.stop();
    writefln("buffer2 done in usecs: %s.", sw.peek().usecs);
}

The actual initialization is much faster for vector ops than using the foreach
loop, but that's not what is being tested here.
May 10 2011
parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Sorry, I don't this has anything to do with vector ops. Concatenation
always triggers memory allocation for new arrays, right? This could
explain the slowdown. In fact if I just test the performance of the
foreach-initialized buffer over 50 runs, I get results like:

buffer done in usecs: 608.
buffer done in usecs: 604.
buffer done in usecs: 607.
buffer done in usecs: 604.
buffer done in usecs: 608.
buffer done in usecs: 614.
buffer done in usecs: 613.
buffer done in usecs: 601.
buffer done in usecs: 596.
buffer done in usecs: 606.
buffer done in usecs: 607.
buffer done in usecs: 24353. <- boom

So this probably has something to do with memory and paging. :)
May 10 2011