www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.announce - Short article on std.parallism

reply "Jeremy Wright" <jeremy codestrokes.com> writes:
I implemented bucket sort in D to demonstrate how easy it is to use 
std.parallelism.  I welcome any feedback.

http://www.codestrokes.com/archives/116

Jeremy Wright
May 29 2011
next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Very cool article. :)

Btw, you can omit 'auto' when you use 'immutable' declarations.
May 29 2011
prev sibling next sibling parent "Lars T. Kyllingstad" <public kyllingen.NOSPAMnet> writes:
On Sun, 29 May 2011 18:18:14 -0700, Jeremy Wright wrote:


 I implemented bucket sort in D to demonstrate how easy it is to use
 std.parallelism.  I welcome any feedback.
 
 http://www.codestrokes.com/archives/116


Nice article. :)

A tip to make the code even more terse:  You can replace taskPool.parallel
() with parallel().  The latter just forwards to the former.  (See the 
last item of the std.parallelism docs.)

-Lars
May 30 2011
prev sibling next sibling parent Johann MacDonagh <johann.macdonagh..no spam..gmail.com> writes:
On 5/29/2011 9:18 PM, Jeremy Wright wrote:

 I implemented bucket sort in D to demonstrate how easy it is to use
 std.parallelism. I welcome any feedback.

 http://www.codestrokes.com/archives/116

 Jeremy Wright


Nice. Just a few nit-picks:

1. End of first paragraph: "makes writing parallel code, correct." -> 
"makes writing parallel code correct."

2. Middle of second paragraph: "The tools are designed to find 
deadlocks, analyze timing". You wrote that line in the paragraph above. 
Seemed a little clunky for me.

3. The light green used in the chart and the D code is difficult to see 
on the white background.

4. Maybe a little comment just before the version(MultiThreaded) line to 
show readers that this is where the magic happens. Helps for readers new 
to D so they're not stumbling over the initial bucket creation code.

5. Maybe a comment at the end talking about std.parallel_algorithm. I 
suppose this wouldn't be a good idea if the idea gets dropped or 
radically changed, but even a little comment mentioning how 
std.parallelism will be wrapped for easy algorithm usage in the future 
would be a good selling point.

Anyway, great article!
May 30 2011
prev sibling next sibling parent Graham Fawcett <fawcett uwindsor.ca> writes:
On Sun, 29 May 2011 18:18:14 -0700, Jeremy Wright wrote:


 I implemented bucket sort in D to demonstrate how easy it is to use
 std.parallelism.  I welcome any feedback.
 
 http://www.codestrokes.com/archives/116


 

Haven't read it yet, but:

"like many faucets" --> "like many facets"

Best,
Graham
May 30 2011
prev sibling next sibling parent "Jeremy Wright" <jeremy codestrokes.com> writes:
Wow! Thank you for your feedback. I'll work through your comments. I 
appreciate you all taking the time to read my article.

Jeremy Wright

"Jeremy Wright"  wrote in message news:irurgr$1g65$1 digitalmars.com...

I implemented bucket sort in D to demonstrate how easy it is to use
std.parallelism.  I welcome any feedback.

http://www.codestrokes.com/archives/116

Jeremy Wright
May 30 2011
prev sibling next sibling parent "Jeremy Wright" <jeremy codestrokes.com> writes:
Wow! Thank you for your feedback. I'll work through your comments. I 
appreciate you all taking the time to read my article.

Jeremy Wright

"Jeremy Wright"  wrote in message news:irurgr$1g65$1 digitalmars.com...

I implemented bucket sort in D to demonstrate how easy it is to use
std.parallelism.  I welcome any feedback.

http://www.codestrokes.com/archives/116

Jeremy Wright
May 30 2011
prev sibling next sibling parent reply "Vladimir Panteleev" <vladimir thecybershadow.net> writes:
On Mon, 30 May 2011 04:18:14 +0300, Jeremy Wright <jeremy codestrokes.com>  
wrote:


 I implemented bucket sort in D to demonstrate how easy it is to use  
 std.parallelism.  I welcome any feedback.


One thing: I would suggest to avoid using ~= in a tight loop, as it is  
rather slow. Using std.array.appender for the first loop and  
std.array.join for the second one should be much faster.

-- 
Best regards,
  Vladimir                            mailto:vladimir thecybershadow.net
May 30 2011
next sibling parent Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
On 5/30/11, Vladimir Panteleev <vladimir thecybershadow.net> wrote:

 On Mon, 30 May 2011 04:18:14 +0300, Jeremy Wright <jeremy codestrokes.com>
 wrote:


 I implemented bucket sort in D to demonstrate how easy it is to use
 std.parallelism.  I welcome any feedback.


 One thing: I would suggest to avoid using ~= in a tight loop, as it is
 rather slow. Using std.array.appender for the first loop and
 std.array.join for the second one should be much faster.

 --
 Best regards,
   Vladimir                            mailto:vladimir thecybershadow.net


I wonder why we even have this operator in the language if we're
supposed to avoid it all the time.
May 30 2011
prev sibling next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On 2011-05-30 22:52, Andrej Mitrovic wrote:

 On 5/30/11, Vladimir Panteleev <vladimir thecybershadow.net> wrote:

 On Mon, 30 May 2011 04:18:14 +0300, Jeremy Wright
 <jeremy codestrokes.com>
 
 wrote:

 I implemented bucket sort in D to demonstrate how easy it is to use
 std.parallelism.  I welcome any feedback.


 
 One thing: I would suggest to avoid using ~= in a tight loop, as it is
 rather slow. Using std.array.appender for the first loop and
 std.array.join for the second one should be much faster.
 
 --
 Best regards,
 
   Vladimir                            mailto:vladimir thecybershadow.net


 
 I wonder why we even have this operator in the language if we're
 supposed to avoid it all the time.


There's no problem with using it, generally-speaking. And it's actually faster 
than it used to be, so it's less of an issue. The problem really only lines 
with a tight loop. Extra checks have to be made with ~= to verify that 
reallocation isn't needed which don't have to be made with Appender, because 
it's able to make assumptions that ~= can't make. So, if you're in a loop that 
appends over and over for a large number of iterations, it's going to be 
faster to use Appender. Other than that, ~= is plenty fast enough.

- Jonathan M Davis
May 30 2011
prev sibling parent reply Andrej Mitrovic <andrej.mitrovich gmail.com> writes:
Why doesn't Appender overload opCatAssign? It would be almost trivial
to replace usage of existing arrays with Appender, instead of having
to replace all calls with var.put().

And why doesn't it overload toString? You can't print its contents to
stdout like you can with slices.

And why can't you slice an Appender?

I see a lot of drawbacks with the only benefit being performance. Now,
if Appender had some syntax sugar to make it appear as if it were a
simple dynamic array (well, slice..), that would be sweet and would
encourage its use, at least with me. Otherwise it just looks ugly
compared to the sexy D arrays. It looks like everything is becoming a
template in a library these days..
May 30 2011
next sibling parent "Steven Schveighoffer" <schveiguy yahoo.com> writes:
On Tue, 31 May 2011 02:21:13 -0400, Andrej Mitrovic  
<andrej.mitrovich gmail.com> wrote:


 Why doesn't Appender overload opCatAssign? It would be almost trivial
 to replace usage of existing arrays with Appender, instead of having
 to replace all calls with var.put().


It should, there might already be an enhancement filed for it.


 And why doesn't it overload toString? You can't print its contents to
 stdout like you can with slices.

 And why can't you slice an Appender?


writeln(app.data);
auto slice = app.data[2...6];

-Steve
May 31 2011
prev sibling parent "Robert Jacques" <sandford jhu.edu> writes:
On Tue, 31 May 2011 02:21:13 -0400, Andrej Mitrovic  
<andrej.mitrovich gmail.com> wrote:

 Why doesn't Appender overload opCatAssign? It would be almost trivial
 to replace usage of existing arrays with Appender, instead of having
 to replace all calls with var.put().


I've submitted a patch for an improved appender implementation. I didn't  
include '~=' as at the time, I thought it unnecessarily complicated  
dependent patch to std.range.put. ( See:  
http://d.puremagic.com/issues/show_bug.cgi?id=5813 and  
http://d.puremagic.com/issues/show_bug.cgi?id=5233 ) On review, I think it  
should be a trivial fix. And, as there were two enhancement requests in  
bugzilla for appender + "~=" (See Issue 4287), I will update my patches.


 And why doesn't it overload toString? You can't print its contents to
 stdout like you can with slices.


In the current appender? No reason. In my patched appender? Less trivial  
but doable. I've put it on my todo list.


 And why can't you slice an Appender?


What do you mean by a slice? An appender slice is non-nonsensical, since  
it can't append. If you mean slicing an appender to get a T[]; there is no  
technical reason why it can't be done now. But it's bad design. Supporting  
slicing, indexing and even length all expose too much of the underlying  
implementation. And array building utilities should be allowed to change  
their internal implementations. For example, in .Net 4.0, StringBuilder  
changed from a dynamic array to a linked list. And a linked list can't  
efficiently index nor slice. Regarding my patched appender, O(1) indexing  
and slicing is doable, but would require some additional memory overhead  
and would not be as fast as built-in arrays.


 I see a lot of drawbacks with the only benefit being performance.


It also lets you build immutable and const arrays like they were mutable.  
Oh, and for my patch that performance gain, in terms of free memory and  
performance is massive for large arrays.


 Now,
 if Appender had some syntax sugar to make it appear as if it were a
 simple dynamic array (well, slice..), that would be sweet and would
 encourage its use, at least with me. Otherwise it just looks ugly
 compared to the sexy D arrays. It looks like everything is becoming a
 template in a library these days..


A good array builder _isn't_ a simple array (dynamic or otherwise) and it  
shouldn't be used as such. Appender has one job and needs to do it well.  
Supporting other functions that have very poor performance is a sure way  
to whittle away all the performance gains using Appender gets you in the  
first place.
Jun 08 2011
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On 2011-05-30 23:21, Andrej Mitrovic wrote:

 Why doesn't Appender overload opCatAssign? It would be almost trivial
 to replace usage of existing arrays with Appender, instead of having
 to replace all calls with var.put().
 
 And why doesn't it overload toString? You can't print its contents to
 stdout like you can with slices.
 
 And why can't you slice an Appender?
 
 I see a lot of drawbacks with the only benefit being performance. Now,
 if Appender had some syntax sugar to make it appear as if it were a
 simple dynamic array (well, slice..), that would be sweet and would
 encourage its use, at least with me. Otherwise it just looks ugly
 compared to the sexy D arrays. It looks like everything is becoming a
 template in a library these days..


If you could slice an Appender, it would lose all of its guarantees. It can do 
what it does in part because it knows that it hasn't been sliced. If you were 
slicing it, then you'd have to worry about possible reallocations which loses 
all of the benefit of Appender in the first place. Appender only works because 
it's creating an array. If you tried to use it as a dynamic array in general, 
it wouldn't work. It's purely an optimization for creating arrays. ~= can't be 
made to do what Appender does, and if you tried to make Appender do what ~= 
does, then Appender couldn't do what it currently does. Appender has a very 
specific purpose and no one should be trying to replace dynamic arrays with 
it. It's just that if you want to get your code to be as fast as possible, 
there are cases where using Appender is a good idea. Beyond that, you 
shouldn't be using Appender.

- Jonathan M Davis
May 30 2011