www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Running Phobos unit tests in threads: I have data

reply "Atila Neves" <atila.neves gmail.com> writes:
So I tried using unit-threaded to run Phobos unit tests again and 
had problems (which I'll look into later) with its compile-time 
reflection. Then I realised I was an idiot since I don't need to 
reflect on anything: all Phobos tests are in unittest blocks so 
all I need to do is include them in the build and unit-threaded 
will run them for me.

I tried a basic sanity check by running them in one thread only 
with the -s option and got a segfault, and a failing test before 
that. None of this should happen, and I'll be taking a look at 
that as well.

But I carried on by removing the troublesome modules from the 
build. These turned out to be:

std.datetime (fails)
std.process (fails and causes the segfault)
std.stdio (fails)

All the others pass in single threaded mode. After this I tried 
using threads and std.parallelism failed, so I took that away 
from the build as well.

Another thing to mention is that although the tests are running 
in threads, since when I wrote the library the getUnitTests 
__traits wasn't available (and since then I wasn't interested in 
using it), each module's unit tests run as one test. So they only 
interleave with other modules, not with each other.

Running in one thread took 39 +/- 1 seconds.
Running in 8 threads took... ~41 seconds.

Oops. I noticed some tests take a lot longer so I tried removing 
those. They were:

std.file
std.conv
std.regex
std.random
std.container
std.xml
std.utf
std.numeric
std.uuid
std.exception

I also removed any modules that were likely to be problematic 
like std.concurrency and std.socket. With the reduced sample size 
the results were:

1 thread: ~1.9s
8 threads: 4.1s +/- 0.2

So the whole threading thing isn't looking so great. Or at least 
not how I implemented it. This got me thinking about my own 
projects. The tests run so fast I never really paid attention to 
how fast they were running. I compared running the unit tests in 
Cerealed in one or more threads and got the same result: running 
in one thread was faster.

I have to look to be sure but maybe the bottleneck is output. As 
in actually printing the results to the screen. I had to jump 
through a few hoops to make sure the output wasn't interleaved, 
and in the end decided to have one thread be responsible for 
that, with the tests sending it output messages.

For reference, I copied all of the std/*.d modules into a local 
std directory, compiled all of them with dmd -unittest -c, then 
used this as the build command:

dmd -unittest -I~/coding/d/unit-threaded/source ut.d 
std/algorithm.o std/array.o std/ascii.o std/base64.o std/bigint.o 
std/bitmanip.o std/compiler.o std/complex.o std/container.o 
std/cstream.o std/csv.o std/demangle.o std/encoding.o 
std/format.o std/functional.o std/getopt.o std/json.o std/math.o 
std/mathspecial.o std/metastrings.o std/mmfile.o std/numeric.o 
std/outbuffer.o std/range.o  std/signals.o  std/stdint.o 
std/stdiobase.o std/stream.o std/string.o std/syserror.o 
std/system.o std/traits.o std/typecons.o std/typelist.o 
std/typetuple.o std/uri.o std/variant.o std/zip.o std/zlib.o  
libunit-threaded.a -ofphobos_ut

I got libunit-threaded.a by running "dub build" in the root 
directory of unit-threaded.

I might just implement a random order option now. Hmm.

Atila
May 03 2014
next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
I turned off all output to check. It was still slower with 
multiple threads. That was the only "weird" thing I was doing I 
could think of as the cause. Otherwise it's just a foreach(test; 
tests.parallel) { test(); }.

Atila

On Saturday, 3 May 2014 at 11:54:55 UTC, Atila Neves wrote:

 So I tried using unit-threaded to run Phobos unit tests again 
 and had problems (which I'll look into later) with its 
 compile-time reflection. Then I realised I was an idiot since I 
 don't need to reflect on anything: all Phobos tests are in 
 unittest blocks so all I need to do is include them in the 
 build and unit-threaded will run them for me.

 I tried a basic sanity check by running them in one thread only 
 with the -s option and got a segfault, and a failing test 
 before that. None of this should happen, and I'll be taking a 
 look at that as well.

 But I carried on by removing the troublesome modules from the 
 build. These turned out to be:

 std.datetime (fails)
 std.process (fails and causes the segfault)
 std.stdio (fails)

 All the others pass in single threaded mode. After this I tried 
 using threads and std.parallelism failed, so I took that away 
 from the build as well.

 Another thing to mention is that although the tests are running 
 in threads, since when I wrote the library the getUnitTests 
 __traits wasn't available (and since then I wasn't interested 
 in using it), each module's unit tests run as one test. So they 
 only interleave with other modules, not with each other.

 Running in one thread took 39 +/- 1 seconds.
 Running in 8 threads took... ~41 seconds.

 Oops. I noticed some tests take a lot longer so I tried 
 removing those. They were:

 std.file
 std.conv
 std.regex
 std.random
 std.container
 std.xml
 std.utf
 std.numeric
 std.uuid
 std.exception

 I also removed any modules that were likely to be problematic 
 like std.concurrency and std.socket. With the reduced sample 
 size the results were:

 1 thread: ~1.9s
 8 threads: 4.1s +/- 0.2

 So the whole threading thing isn't looking so great. Or at 
 least not how I implemented it. This got me thinking about my 
 own projects. The tests run so fast I never really paid 
 attention to how fast they were running. I compared running the 
 unit tests in Cerealed in one or more threads and got the same 
 result: running in one thread was faster.

 I have to look to be sure but maybe the bottleneck is output. 
 As in actually printing the results to the screen. I had to 
 jump through a few hoops to make sure the output wasn't 
 interleaved, and in the end decided to have one thread be 
 responsible for that, with the tests sending it output messages.

 For reference, I copied all of the std/*.d modules into a local 
 std directory, compiled all of them with dmd -unittest -c, then 
 used this as the build command:

 dmd -unittest -I~/coding/d/unit-threaded/source ut.d 
 std/algorithm.o std/array.o std/ascii.o std/base64.o 
 std/bigint.o std/bitmanip.o std/compiler.o std/complex.o 
 std/container.o std/cstream.o std/csv.o std/demangle.o 
 std/encoding.o std/format.o std/functional.o std/getopt.o 
 std/json.o std/math.o std/mathspecial.o std/metastrings.o 
 std/mmfile.o std/numeric.o std/outbuffer.o std/range.o  
 std/signals.o  std/stdint.o std/stdiobase.o std/stream.o 
 std/string.o std/syserror.o std/system.o std/traits.o 
 std/typecons.o std/typelist.o std/typetuple.o std/uri.o 
 std/variant.o std/zip.o std/zlib.o  libunit-threaded.a 
 -ofphobos_ut

 I got libunit-threaded.a by running "dub build" in the root 
 directory of unit-threaded.

 I might just implement a random order option now. Hmm.

 Atila
May 03 2014
parent reply "Rikki Cattermole" <alphaglosined gmail.com> writes:
On Saturday, 3 May 2014 at 12:08:56 UTC, Atila Neves wrote:

 I turned off all output to check. It was still slower with 
 multiple threads. That was the only "weird" thing I was doing I 
 could think of as the cause. Otherwise it's just a 
 foreach(test; tests.parallel) { test(); }.

 Atila

 On Saturday, 3 May 2014 at 11:54:55 UTC, Atila Neves wrote:

 So I tried using unit-threaded to run Phobos unit tests again 
 and had problems (which I'll look into later) with its 
 compile-time reflection. Then I realised I was an idiot since 
 I don't need to reflect on anything: all Phobos tests are in 
 unittest blocks so all I need to do is include them in the 
 build and unit-threaded will run them for me.

 I tried a basic sanity check by running them in one thread 
 only with the -s option and got a segfault, and a failing test 
 before that. None of this should happen, and I'll be taking a 
 look at that as well.

 But I carried on by removing the troublesome modules from the 
 build. These turned out to be:

 std.datetime (fails)
 std.process (fails and causes the segfault)
 std.stdio (fails)

 All the others pass in single threaded mode. After this I 
 tried using threads and std.parallelism failed, so I took that 
 away from the build as well.

 Another thing to mention is that although the tests are 
 running in threads, since when I wrote the library the 
 getUnitTests __traits wasn't available (and since then I 
 wasn't interested in using it), each module's unit tests run 
 as one test. So they only interleave with other modules, not 
 with each other.

 Running in one thread took 39 +/- 1 seconds.
 Running in 8 threads took... ~41 seconds.

 Oops. I noticed some tests take a lot longer so I tried 
 removing those. They were:

 std.file
 std.conv
 std.regex
 std.random
 std.container
 std.xml
 std.utf
 std.numeric
 std.uuid
 std.exception

 I also removed any modules that were likely to be problematic 
 like std.concurrency and std.socket. With the reduced sample 
 size the results were:

 1 thread: ~1.9s
 8 threads: 4.1s +/- 0.2

 So the whole threading thing isn't looking so great. Or at 
 least not how I implemented it. This got me thinking about my 
 own projects. The tests run so fast I never really paid 
 attention to how fast they were running. I compared running 
 the unit tests in Cerealed in one or more threads and got the 
 same result: running in one thread was faster.

 I have to look to be sure but maybe the bottleneck is output. 
 As in actually printing the results to the screen. I had to 
 jump through a few hoops to make sure the output wasn't 
 interleaved, and in the end decided to have one thread be 
 responsible for that, with the tests sending it output 
 messages.

 For reference, I copied all of the std/*.d modules into a 
 local std directory, compiled all of them with dmd -unittest 
 -c, then used this as the build command:

 dmd -unittest -I~/coding/d/unit-threaded/source ut.d 
 std/algorithm.o std/array.o std/ascii.o std/base64.o 
 std/bigint.o std/bitmanip.o std/compiler.o std/complex.o 
 std/container.o std/cstream.o std/csv.o std/demangle.o 
 std/encoding.o std/format.o std/functional.o std/getopt.o 
 std/json.o std/math.o std/mathspecial.o std/metastrings.o 
 std/mmfile.o std/numeric.o std/outbuffer.o std/range.o  
 std/signals.o  std/stdint.o std/stdiobase.o std/stream.o 
 std/string.o std/syserror.o std/system.o std/traits.o 
 std/typecons.o std/typelist.o std/typetuple.o std/uri.o 
 std/variant.o std/zip.o std/zlib.o  libunit-threaded.a 
 -ofphobos_ut

 I got libunit-threaded.a by running "dub build" in the root 
 directory of unit-threaded.

 I might just implement a random order option now. Hmm.

 Atila




Out of curiosity are you on Windows?
May 03 2014
parent reply "Atila Neves" <atila.neves gmail.com> writes:
 Out of curiosity are you on Windows?


No, Arch Linux 64-bit. I also just noticed a glaring threading 
bug in my code as well that somehow's never turned up. This is 
not a good day.

Atila
May 03 2014
parent reply "Rikki Cattermole" <alphaglosined gmail.com> writes:
On Saturday, 3 May 2014 at 12:24:59 UTC, Atila Neves wrote:

 Out of curiosity are you on Windows?


 No, Arch Linux 64-bit. I also just noticed a glaring threading 
 bug in my code as well that somehow's never turned up. This is 
 not a good day.

 Atila


I'm surprised. Threads should be cheap on Linux. Something funky 
is definitely going on I bet.
May 03 2014
next sibling parent Walter Bright <newshound2 digitalmars.com> writes:
On 5/3/2014 5:26 AM, Rikki Cattermole wrote:

 Something funky is definitely going on I bet.


No doubt: http://www.youtube.com/watch?v=aZcbDESaxhY
May 03 2014
prev sibling parent reply "Dicebot" <public dicebot.lv> writes:
On Saturday, 3 May 2014 at 12:26:13 UTC, Rikki Cattermole wrote:

 On Saturday, 3 May 2014 at 12:24:59 UTC, Atila Neves wrote:

 Out of curiosity are you on Windows?


 No, Arch Linux 64-bit. I also just noticed a glaring threading 
 bug in my code as well that somehow's never turned up. This is 
 not a good day.

 Atila


 I'm surprised. Threads should be cheap on Linux. Something 
 funky is definitely going on I bet.


Threads are never cheap.
May 05 2014
parent "Brad Anderson" <eco gnuk.net> writes:
On Monday, 5 May 2014 at 17:56:11 UTC, Dicebot wrote:

 On Saturday, 3 May 2014 at 12:26:13 UTC, Rikki Cattermole wrote:

 On Saturday, 3 May 2014 at 12:24:59 UTC, Atila Neves wrote:

 Out of curiosity are you on Windows?


 No, Arch Linux 64-bit. I also just noticed a glaring 
 threading bug in my code as well that somehow's never turned 
 up. This is not a good day.

 Atila


 I'm surprised. Threads should be cheap on Linux. Something 
 funky is definitely going on I bet.


 Threads are never cheap.


Regarding this, I found this talk interesting: 
https://www.youtube.com/watch?v=KXuZi9aeGTw
May 05 2014
prev sibling next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
Ok, so I went and added __traits(getUnitTests) to unit-threaded. 
That way each unittest block is its own test case. I registered 
these modules in std to run:

array, ascii, base64, bigint, bitmanip, concurrency, container, 
cstream.

On the good news front, they all passed even though they were 
running concurrently.

On the bad news front, single-threaded operation was still faster 
(0.22s vs 0.28s). I still don't know why.

I fixed my concurrency bug, now I'm using taskPool.amap.


Atila
May 03 2014
parent reply "Atila Neves" <atila.neves gmail.com> writes:
I can reproduce the slower-with-threads issue without using my 
library. I've included the source file below and would like to 
know if other people see the same thing.

The Phobos modules are all called "ustd" because I 
couldn't/didn't know how to get this to work otherwise. So I 
copied the std/*.d files to a directory called ustd and changed 
their module declarations. Silly but it works. I'd love to know 
how to do this properly.

With this file, I consistenly get faster times with "-s" (for 
single-threaded) than without (multi-threaded):


import std.parallelism;
import std.getopt;


import ustd.array;
import ustd.ascii;
import ustd.base64;
import ustd.bigint;
import ustd.bitmanip;
import ustd.concurrency;
import ustd.container;
import ustd.cstream;


alias TestFunction = void function();

auto getTests(Modules...)() {
     TestFunction[] tests;
     foreach(mod; Modules) {
         foreach(test; __traits(getUnitTests, mod)) {
             tests ~= &test;
         }
     }
     return tests;
}



void main(string[] args) {
     bool single;
     getopt(args,
            "single|s", &single
         );

     enum tests = getTests!(
         ustd.array,
         ustd.ascii,
         ustd.base64,
         ustd.bigint,
         ustd.bitmanip,
         ustd.concurrency,
         ustd.container,
         ustd.cstream,
         );

     if(single) {
         foreach(test; tests) {
             test();
         }
     } else {
         foreach(test; tests.parallel) {
             test();
         }
     }
}
May 03 2014
next sibling parent reply Walter Bright <newshound2 digitalmars.com> writes:
On 5/3/2014 10:22 AM, Atila Neves wrote:

 I can reproduce the slower-with-threads issue without using my library. I've
 included the source file below and would like to know if other people see the
 same thing.


I haven't investigated this, but my suspicions are:

1. thread creation/destruction is dominating the times.

2. since very few of the unittests block, there is no speed advantage from 
having more threads than cores. If you limit the number of threads to the
number 
of cores on your machine, you might see a speedup.
May 03 2014
parent "Atila Neves" <atila.neves gmail.com> writes:
On Saturday, 3 May 2014 at 18:26:37 UTC, Walter Bright wrote:

 On 5/3/2014 10:22 AM, Atila Neves wrote:

 I can reproduce the slower-with-threads issue without using my 
 library. I've
 included the source file below and would like to know if other 
 people see the
 same thing.


 I haven't investigated this, but my suspicions are:

 1. thread creation/destruction is dominating the times.


In the current measurements probably since the whole run takes 
less than a second. But the first ones I did were dozens of 
seconds long, so I don't think so.


 2. since very few of the unittests block, there is no speed 
 advantage from having more threads than cores. If you limit the 
 number of threads to the number of cores on your machine, you 
 might see a speedup.


Like I mentioned above, unless I'm mistaken taskPool should be 
using a correct number of threads for my machine already.
May 03 2014
prev sibling parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
03-May-2014 21:22, Atila Neves пишет:

 I can reproduce the slower-with-threads issue without using my library.
 I've included the source file below and would like to know if other
 people see the same thing.

 The Phobos modules are all called "ustd" because I couldn't/didn't know
 how to get this to work otherwise. So I copied the std/*.d files to a
 directory called ustd and changed their module declarations. Silly but
 it works. I'd love to know how to do this properly.


[snip]


      if(single) {
          foreach(test; tests) {
              test();
          }
      } else {
          foreach(test; tests.parallel) {


Try different batch size:
test.parallel(1), test.parallel(2) etc.



-- 
Dmitry Olshansky
May 03 2014
parent reply "Atila Neves" <atila.neves gmail.com> writes:
     if(single) {
         foreach(test; tests) {
             test();
         }
     } else {
         foreach(test; tests.parallel) {


 Try different batch size:
 test.parallel(1), test.parallel(2) etc.


So as to not have thread creation be disproportionately 
represented, I repeated the module list over and over again, 
making the number of tests run equal to 9990. This takes 5s on my 
machine to run in on thread and 12s in multiple. Here are the 
things I tried:

1. Created my own TaskPool so I could decide how many threads to 
use
2. Changed the batch size in parallel from 1 to 10 to 100 to 1000
3. Explicitly spawn two threads and tell each to do a foreach on 
half of the tests


None of them made it go any faster. I had similar results using 
unit-threaded on my own projects. This is weird.

Atila
May 03 2014
parent reply "Atila Neves" <atila.neves gmail.com> writes:
gdc gave _very_ different results. I had to use different modules 
because at some point tests started failing, but with gdc the 
threaded version runs ~3x faster.

On my own unit-threaded benchmarks, running the UTs for Cerealed 
over and over again was only slightly slower with threads than 
without. With dmd the threaded version was nearly 3x slower.

Atila

On Saturday, 3 May 2014 at 21:14:29 UTC, Atila Neves wrote:

    if(single) {
        foreach(test; tests) {
            test();
        }
    } else {
        foreach(test; tests.parallel) {


 Try different batch size:
 test.parallel(1), test.parallel(2) etc.


 So as to not have thread creation be disproportionately 
 represented, I repeated the module list over and over again, 
 making the number of tests run equal to 9990. This takes 5s on 
 my machine to run in on thread and 12s in multiple. Here are 
 the things I tried:

 1. Created my own TaskPool so I could decide how many threads 
 to use
 2. Changed the batch size in parallel from 1 to 10 to 100 to 
 1000
 3. Explicitly spawn two threads and tell each to do a foreach 
 on half of the tests


 None of them made it go any faster. I had similar results using 
 unit-threaded on my own projects. This is weird.

 Atila
May 03 2014
next sibling parent "Atila Neves" <atila.neves gmail.com> writes:
Same thing with unit_threaded on Phobos, 3x faster even without 
repeating the modules (0.1s vs 0.3s). Since the example is 
shorter than the other one, I'll post it here in case anyone else 
wants to try:

import unit_threaded.runner;

int main(string[] args) {
     return args.runTests!(
         "ustd.array",
         "ustd.ascii",
         "ustd.base64",
         "ustd.bigint",
         "ustd.bitmanip",
         "ustd.concurrency",
         "ustd.container",
         "ustd.cstream",
         );
}


On Saturday, 3 May 2014 at 21:42:13 UTC, Atila Neves wrote:

 gdc gave _very_ different results. I had to use different 
 modules because at some point tests started failing, but with 
 gdc the threaded version runs ~3x faster.

 On my own unit-threaded benchmarks, running the UTs for 
 Cerealed over and over again was only slightly slower with 
 threads than without. With dmd the threaded version was nearly 
 3x slower.

 Atila

 On Saturday, 3 May 2014 at 21:14:29 UTC, Atila Neves wrote:

   if(single) {
       foreach(test; tests) {
           test();
       }
   } else {
       foreach(test; tests.parallel) {


 Try different batch size:
 test.parallel(1), test.parallel(2) etc.


 So as to not have thread creation be disproportionately 
 represented, I repeated the module list over and over again, 
 making the number of tests run equal to 9990. This takes 5s on 
 my machine to run in on thread and 12s in multiple. Here are 
 the things I tried:

 1. Created my own TaskPool so I could decide how many threads 
 to use
 2. Changed the batch size in parallel from 1 to 10 to 100 to 
 1000
 3. Explicitly spawn two threads and tell each to do a foreach 
 on half of the tests


 None of them made it go any faster. I had similar results 
 using unit-threaded on my own projects. This is weird.

 Atila
May 03 2014
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/3/14, 2:42 PM, Atila Neves wrote:

 gdc gave _very_ different results. I had to use different modules
 because at some point tests started failing, but with gdc the threaded
 version runs ~3x faster.

 On my own unit-threaded benchmarks, running the UTs for Cerealed over
 and over again was only slightly slower with threads than without. With
 dmd the threaded version was nearly 3x slower.


Sounds like a severe bug in dmd or dependents. -- Andrei
May 03 2014
next sibling parent reply "Atila Neves" <atila.neves gmail.com> writes:
On Saturday, 3 May 2014 at 22:46:03 UTC, Andrei Alexandrescu 
wrote:

 On 5/3/14, 2:42 PM, Atila Neves wrote:

 gdc gave _very_ different results. I had to use different 
 modules
 because at some point tests started failing, but with gdc the 
 threaded
 version runs ~3x faster.

 On my own unit-threaded benchmarks, running the UTs for 
 Cerealed over
 and over again was only slightly slower with threads than 
 without. With
 dmd the threaded version was nearly 3x slower.


 Sounds like a severe bug in dmd or dependents. -- Andrei


Seems like it. Just to be sure I swapped ld.gold for ld.bfd and 
the problem was still there.

I'm not entirely sure how to file this bug: with just my simple 
example above?

Atila
May 04 2014
parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/4/14, 1:44 AM, Atila Neves wrote:

 On Saturday, 3 May 2014 at 22:46:03 UTC, Andrei Alexandrescu wrote:

 On 5/3/14, 2:42 PM, Atila Neves wrote:

 gdc gave _very_ different results. I had to use different modules
 because at some point tests started failing, but with gdc the threaded
 version runs ~3x faster.

 On my own unit-threaded benchmarks, running the UTs for Cerealed over
 and over again was only slightly slower with threads than without. With
 dmd the threaded version was nearly 3x slower.


 Sounds like a severe bug in dmd or dependents. -- Andrei


 Seems like it. Just to be sure I swapped ld.gold for ld.bfd and the
 problem was still there.

 I'm not entirely sure how to file this bug: with just my simple example
 above?


The simpler the better. -- Andrei
May 04 2014
parent "Atila Neves" <atila.neves gmail.com> writes:
https://issues.dlang.org/show_bug.cgi?id=12708

On Sunday, 4 May 2014 at 16:07:30 UTC, Andrei Alexandrescu wrote:

 On 5/4/14, 1:44 AM, Atila Neves wrote:

 On Saturday, 3 May 2014 at 22:46:03 UTC, Andrei Alexandrescu 
 wrote:

 On 5/3/14, 2:42 PM, Atila Neves wrote:

 gdc gave _very_ different results. I had to use different 
 modules
 because at some point tests started failing, but with gdc 
 the threaded
 version runs ~3x faster.

 On my own unit-threaded benchmarks, running the UTs for 
 Cerealed over
 and over again was only slightly slower with threads than 
 without. With
 dmd the threaded version was nearly 3x slower.


 Sounds like a severe bug in dmd or dependents. -- Andrei


 Seems like it. Just to be sure I swapped ld.gold for ld.bfd 
 and the
 problem was still there.

 I'm not entirely sure how to file this bug: with just my 
 simple example
 above?


 The simpler the better. -- Andrei
May 05 2014
prev sibling parent reply "safety0ff" <safety0ff.dev gmail.com> writes:
On Saturday, 3 May 2014 at 22:46:03 UTC, Andrei Alexandrescu 
wrote:

 On 5/3/14, 2:42 PM, Atila Neves wrote:

 gdc gave _very_ different results. I had to use different 
 modules
 because at some point tests started failing, but with gdc the 
 threaded
 version runs ~3x faster.

 On my own unit-threaded benchmarks, running the UTs for 
 Cerealed over
 and over again was only slightly slower with threads than 
 without. With
 dmd the threaded version was nearly 3x slower.


 Sounds like a severe bug in dmd or dependents. -- Andrei


This reminds me of when I was parallelizing a project euler 
solution: atomic access was so much slower on DMD that it made 
performance worse than the single threaded version for one stage 
of the program.

I know that std.parallelism does make use of core.atomic under 
the hood, so this may be a factor when using DMD.
May 04 2014
parent reply "Atila Neves" <atila.neves gmail.com> writes:
On Sunday, 4 May 2014 at 17:01:23 UTC, safety0ff wrote:

 On Saturday, 3 May 2014 at 22:46:03 UTC, Andrei Alexandrescu 
 wrote:

 On 5/3/14, 2:42 PM, Atila Neves wrote:

 gdc gave _very_ different results. I had to use different 
 modules
 because at some point tests started failing, but with gdc the 
 threaded
 version runs ~3x faster.

 On my own unit-threaded benchmarks, running the UTs for 
 Cerealed over
 and over again was only slightly slower with threads than 
 without. With
 dmd the threaded version was nearly 3x slower.


 Sounds like a severe bug in dmd or dependents. -- Andrei


 This reminds me of when I was parallelizing a project euler 
 solution: atomic access was so much slower on DMD that it made 
 performance worse than the single threaded version for one 
 stage of the program.

 I know that std.parallelism does make use of core.atomic under 
 the hood, so this may be a factor when using DMD.


Funny you should say that, a friend of mine tried porting a 
lock-free algorithm of his from Java to D a few weeks ago. The D 
version ran 3 orders of magnitude slower. Then I tried gdc and 
ldc on his code. ldc produced code running at around 80% of the 
speed of the Java version, fdc was around 30%. But dmd...
May 05 2014
next sibling parent Orvid King via Digitalmars-d <digitalmars-d puremagic.com> writes:
Going to take a wild guess, but as core.atomic.casImpl will never be
inlined anywhere with DMD, due to it's inline assembly, you have the
cost of building and destroying a stack frame, the cost of passing the
args in, moving them into registers, saving potentially trashed
registers, etc. every time it even attempts to acquire a lock, and the
GC uses a single global lock for just about everything. As you can
imagine, I suspect this is far from optimal, and, if I remember right,
GDC uses intrinsics for the atomic operations.

On 5/5/14, Atila Neves via Digitalmars-d <digitalmars-d puremagic.com> wrote:

 On Sunday, 4 May 2014 at 17:01:23 UTC, safety0ff wrote:

 On Saturday, 3 May 2014 at 22:46:03 UTC, Andrei Alexandrescu
 wrote:

 On 5/3/14, 2:42 PM, Atila Neves wrote:

 gdc gave _very_ different results. I had to use different
 modules
 because at some point tests started failing, but with gdc the
 threaded
 version runs ~3x faster.

 On my own unit-threaded benchmarks, running the UTs for
 Cerealed over
 and over again was only slightly slower with threads than
 without. With
 dmd the threaded version was nearly 3x slower.


 Sounds like a severe bug in dmd or dependents. -- Andrei


 This reminds me of when I was parallelizing a project euler
 solution: atomic access was so much slower on DMD that it made
 performance worse than the single threaded version for one
 stage of the program.

 I know that std.parallelism does make use of core.atomic under
 the hood, so this may be a factor when using DMD.


 Funny you should say that, a friend of mine tried porting a
 lock-free algorithm of his from Java to D a few weeks ago. The D
 version ran 3 orders of magnitude slower. Then I tried gdc and
 ldc on his code. ldc produced code running at around 80% of the
 speed of the Java version, fdc was around 30%. But dmd...
May 05 2014
prev sibling parent Iain Buclaw via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 5 May 2014 19:07, Orvid King via Digitalmars-d
<digitalmars-d puremagic.com> wrote:

 Going to take a wild guess, but as core.atomic.casImpl will never be
 inlined anywhere with DMD, due to it's inline assembly, you have the
 cost of building and destroying a stack frame, the cost of passing the
 args in, moving them into registers, saving potentially trashed
 registers, etc. every time it even attempts to acquire a lock, and the
 GC uses a single global lock for just about everything. As you can
 imagine, I suspect this is far from optimal, and, if I remember right,
 GDC uses intrinsics for the atomic operations.


Aye, and atomic intrinsics though they may be, it could even be
improved by switching over to C++ atomic intrinsics, which map
directly to core.atomics.  :)
May 05 2014
prev sibling parent reply Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/3/14, 4:54 AM, Atila Neves wrote:

 So I tried using unit-threaded to run Phobos unit tests


[snip]

Thanks. Are you using thread pooling (a limited number of threads e.g. 
1.5 * cores running all unittests)? -- Andrei
May 03 2014
parent reply "Atila Neves" <atila.neves gmail.com> writes:
On Saturday, 3 May 2014 at 18:16:52 UTC, Andrei Alexandrescu 
wrote:

 On 5/3/14, 4:54 AM, Atila Neves wrote:

 So I tried using unit-threaded to run Phobos unit tests


 [snip]

 Thanks. Are you using thread pooling (a limited number of 
 threads e.g. 1.5 * cores running all unittests)? -- Andrei


I'm using parallel and taskPool from std.parallelism. I was under 
the impression it gave me a ready-to-use pool with as many 
threads as I have cores.
May 03 2014
next sibling parent reply Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sat, 2014-05-03 at 19:37 +0000, Atila Neves via Digitalmars-d wrote:
[…]

 I'm using parallel and taskPool from std.parallelism. I was under 
 the impression it gave me a ready-to-use pool with as many 
 threads as I have cores.


There is a default, related to the number of cores the OS thinks there
is (*), but you can also set the number manually.  std.parallelism could
do with some work to make it better than it already is.


(*) Physical cores are not necessarily the number reported by the OS due
to core hyperthreads. Quad core no hyperthreads, and dual core, two
hyperthreads per core, both get reported as four processor systems.
However if you benchmark them you get very, very different performance
characteristics.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
May 04 2014
parent reply "Atila Neves" <atila.neves gmail.com> writes:
Like I mentioned afterwards, I tried a different number of 
threads. On my machine, at least, std.parallelism.totalCPUs 
returns 8, the number of virtual cores. As it should.

Atila

On Sunday, 4 May 2014 at 07:49:51 UTC, Russel Winder via 
Digitalmars-d wrote:

 On Sat, 2014-05-03 at 19:37 +0000, Atila Neves via 
 Digitalmars-d wrote:
 […]

 I'm using parallel and taskPool from std.parallelism. I was 
 under the impression it gave me a ready-to-use pool with as 
 many threads as I have cores.


 There is a default, related to the number of cores the OS 
 thinks there
 is (*), but you can also set the number manually.  
 std.parallelism could
 do with some work to make it better than it already is.


 (*) Physical cores are not necessarily the number reported by 
 the OS due
 to core hyperthreads. Quad core no hyperthreads, and dual core, 
 two
 hyperthreads per core, both get reported as four processor 
 systems.
 However if you benchmark them you get very, very different 
 performance
 characteristics.
May 04 2014
parent reply Russel Winder via Digitalmars-d <digitalmars-d puremagic.com> writes:
On Sun, 2014-05-04 at 08:47 +0000, Atila Neves via Digitalmars-d wrote:

 Like I mentioned afterwards, I tried a different number of 
 threads. On my machine, at least, std.parallelism.totalCPUs 
 returns 8, the number of virtual cores. As it should.


If you can create a small example of the problem, and I can remember how
to run std.parallelism as a separate module, I can try and take a look
at this later next week.

-- 
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.net
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
May 04 2014
parent Andrei Alexandrescu <SeeWebsiteForEmail erdani.org> writes:
On 5/4/14, 3:06 AM, Russel Winder via Digitalmars-d wrote:

 On Sun, 2014-05-04 at 08:47 +0000, Atila Neves via Digitalmars-d wrote:

 Like I mentioned afterwards, I tried a different number of
 threads. On my machine, at least, std.parallelism.totalCPUs
 returns 8, the number of virtual cores. As it should.


 If you can create a small example of the problem, and I can remember how
 to run std.parallelism as a separate module, I can try and take a look
 at this later next week.


This is an awesome offer, Russel. Thanks! -- Andrei
May 04 2014
prev sibling parent Joseph Rushton Wakeling via Digitalmars-d <digitalmars-d puremagic.com> writes:
On 04/05/14 09:49, Russel Winder via Digitalmars-d wrote:

 (*) Physical cores are not necessarily the number reported by the OS due
 to core hyperthreads. Quad core no hyperthreads, and dual core, two
 hyperthreads per core, both get reported as four processor systems.
 However if you benchmark them you get very, very different performance
 characteristics.


Yup.  That bit me with a new laptop the first time I tried parallel programming 
with D :-)
May 04 2014