www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Concurrency in D

reply "Minas Mina" <minas_mina1990 hotmail.co.uk> writes:
I have been "playing" latetly with std.concurrency and 
core.thread. I like both, but they are entirely different 
approaches to concurrency.

std.concurrency: Uses message passing. Does not allow passing of 
mutable types.
core.thread: Essentially the same as the approach taken by Java.

1) Is one favoured over the other? I guess std.concurrency would 
be, because it uses messages (and I recently read somewhere that 
immutability + message passing = good thing). Well, my problem 
about is, how can send mutable data to another thread? _Can_ I do 
it? If not, how am I supposed to share data between them? Not 
everything can be immutable right? For example I would like to 
have a thread calculate the sum of the first half of an array 
while another thread would calculate the other half, and I could 
add the two results at the end.

2) What about core.thread? To do proper sync between threads in 
core.thread, semaphore, mutexes and stuff like that are needed. 
Are those "good practice" in D?

3) I have also read a bit about syncronized classes and shared 
classes/structs, altough I didn't understand it too much (well I 
didn't try too much to be honest).

For all those appoaches, which is the preffered?
For all those appoaches, which is the preffered for game 
programming?

Thank you.
Jun 27 2012
next sibling parent reply Adam Burton <adz21c gmail.com> writes:
Minas Mina wrote:


 I have been "playing" latetly with std.concurrency and
 core.thread. I like both, but they are entirely different
 approaches to concurrency.
 
 std.concurrency: Uses message passing. Does not allow passing of
 mutable types.
 core.thread: Essentially the same as the approach taken by Java.
 
 1) Is one favoured over the other? I guess std.concurrency would
 be, because it uses messages (and I recently read somewhere that
 immutability + message passing = good thing).


I've not really got around to doing any concurrent programming in anger with 
D yet but from what I've read message passing is the preferred method. 
Fortunately when it comes to answering your question the concurrency chapter 
for The D Programming Language book (TDPL) was made freely available as a 
preview http://www.informit.com/articles/article.aspx?p=1609144. Hopefully 
that contains your answers :).


 Well, my problem
 about is, how can send mutable data to another thread? _Can_ I do
 it? If not, how am I supposed to share data between them? Not
 everything can be immutable right?


From what I gather this is where the shared keyword typically comes into 
play which should be detailed in the url above.


 For example I would like to
 have a thread calculate the sum of the first half of an array
 while another thread would calculate the other half, and I could
 add the two results at the end.


For this task there is another concurrency module called std.parallelism 
that is designed for this type of thing (afaik). 
http://dlang.org/phobos/std_parallelism.html


 
 2) What about core.thread? To do proper sync between threads in
 core.thread, semaphore, mutexes and stuff like that are needed.
 Are those "good practice" in D?
 
 3) I have also read a bit about syncronized classes and shared
 classes/structs, altough I didn't understand it too much (well I
 didn't try too much to be honest).
 
 For all those appoaches, which is the preffered?


My observation is std.concurrency is being pushed as the preferred default 
method.

 For all those appoaches, which is the preffered for game
 programming?
 
 Thank you.
Jun 27 2012
parent reply =?UTF-8?B?QWxpIMOHZWhyZWxp?= <acehreli yahoo.com> writes:
On 06/27/2012 04:05 PM, Adam Burton wrote:


 For example I would like to
 have a thread calculate the sum of the first half of an array
 while another thread would calculate the other half, and I could
 add the two results at the end.


 For this task there is another concurrency module called std.parallelism
 that is designed for this type of thing (afaik).
 http://dlang.org/phobos/std_parallelism.html


TDPL predates std.parallelism, so it's not in the book. Although the 
module's documentation is straightforward, I have the following page 
which may be more accessible for some:

   http://ddili.org/ders/d.en/parallelism.html

Clicking [Next] on that pages goes to Message Passing Concurrency:

   http://ddili.org/ders/d.en/concurrency.html

Ali

P.S. Both of those pages are incomplete in some ways, which I will 
address soon.
Jun 27 2012
parent reply "Minas Mina" <minas_mina1990 hotmail.co.uk> writes:
On Wednesday, 27 June 2012 at 23:50:17 UTC, Ali Çehreli wrote:

 On 06/27/2012 04:05 PM, Adam Burton wrote:


 For example I would like to
 have a thread calculate the sum of the first half of an array
 while another thread would calculate the other half, and I




 could

 add the two results at the end.


 For this task there is another concurrency module called


 std.parallelism

 that is designed for this type of thing (afaik).
 http://dlang.org/phobos/std_parallelism.html


 TDPL predates std.parallelism, so it's not in the book. 
 Although the module's documentation is straightforward, I have 
 the following page which may be more accessible for some:

   http://ddili.org/ders/d.en/parallelism.html

 Clicking [Next] on that pages goes to Message Passing 
 Concurrency:

   http://ddili.org/ders/d.en/concurrency.html

 Ali

 P.S. Both of those pages are incomplete in some ways, which I 
 will address soon.


Much better than the library reference :)
Jun 28 2012
parent maarten van damme <maartenvd1994 gmail.com> writes:
yeah, I didn't knew std.parallelism was so easy to use. Speed up was
really big for minimal effort.
Jun 28 2012
prev sibling next sibling parent Justin Whear <justin economicmodeling.com> writes:
On Thu, 28 Jun 2012 00:34:50 +0200, Minas Mina wrote:


 I have been "playing" latetly with std.concurrency and core.thread. I
 like both, but they are entirely different approaches to concurrency.
 
 std.concurrency: Uses message passing. Does not allow passing of mutable
 types.
 core.thread: Essentially the same as the approach taken by Java.
 
 1) Is one favoured over the other? I guess std.concurrency would be,
 because it uses messages (and I recently read somewhere that
 immutability + message passing = good thing). Well, my problem about is,
 how can send mutable data to another thread? _Can_ I do it? If not, how
 am I supposed to share data between them? Not everything can be
 immutable right? For example I would like to have a thread calculate the
 sum of the first half of an array while another thread would calculate
 the other half, and I could add the two results at the end.
 
 2) What about core.thread? To do proper sync between threads in
 core.thread, semaphore, mutexes and stuff like that are needed. Are
 those "good practice" in D?
 
 3) I have also read a bit about syncronized classes and shared
 classes/structs, altough I didn't understand it too much (well I didn't
 try too much to be honest).
 
 For all those appoaches, which is the preffered?
 For all those appoaches, which is the preffered for game programming?
 
 Thank you.


Concurrency and parallelism are two related, but different concepts. 
Concurrency is when you have different computations conducted 
simultaneously. An example is a modern game engine: you may have a 
rendering thread, a physics simulation, and a networking stack all 
operating at the same time on different cores. Concurrency allows (and 
generally assumes) communication between these parts as they need to 
share information. Message-passing is, as far as I know, pretty much the 
only truly correct way of doing this.

Parallelism is, by contrast, the decomposing of a computation into 
smaller chunks which do not depend on each other. A trivial example is 
incrementing all integers in an array by one--the result of the increment 
computation only relies on the integer being worked on, so the 
computations may be performed in parallel and the results collected.

The std.concurrency and std.parallelism thus exist to solve two different 
problems and are more of an apples-oranges situation.

Justin
Jun 27 2012
prev sibling next sibling parent reply "Nathan M. Swan" <nathanmswan gmail.com> writes:
On Wednesday, 27 June 2012 at 22:34:51 UTC, Minas Mina wrote:

 I have been "playing" latetly with std.concurrency and 
 core.thread. I like both, but they are entirely different 
 approaches to concurrency.


Aren't they great? ;)


 std.concurrency: Uses message passing. Does not allow passing 
 of mutable types.
 core.thread: Essentially the same as the approach taken by Java.

 1) Is one favoured over the other? I guess std.concurrency 
 would be, because it uses messages (and I recently read 
 somewhere that immutability + message passing = good thing).


Personally I favor std.concurrency, in many cases I find it 
faster.


 Well, my problem about is, how can send mutable data to another 
 thread? _Can_ I do it? If not, how am I supposed to share data 
 between them? Not everything can be immutable right?


Mutable value types can be sent easily, because they are copied. 
You share data with shared(T) types, e.g. shared(Object) or 
shared(int[]).

However, I try to avoid shared data by keeping it in single 
threads, and having other threads access parts with message 
passing.

One paradigm is to isolate data into "thread objects," threads 
which have non-shared data on the stack, and have other threads 
send GetData messages and get ReturnedData messages.


 For example I would like to have a thread calculate the sum of 
 the first half of an array while another thread would calculate 
 the other half, and I could add the two results at the end.


Check out this:
http://dpaste.dzfl.pl/0f7ccb79

Note that for algorithms like this, you should check out 
std.parallelism, it's more high-level than std.concurrency.


 2) What about core.thread? To do proper sync between threads in 
 core.thread, semaphore, mutexes and stuff like that are needed. 
 Are those "good practice" in D?


Part of D's philosophy is "you can get low-level and dirty if you 
want, but the easy/safe way should also be the fast way." 
core.thread is there for you, but std.concurrency is easier. Note 
that std.concurrency is written on top of core.thread.

I definitely prefer std.concurrency, but I don't write 
CPU-intensive stuff anyway, so I'm no expert.


 3) I have also read a bit about syncronized classes and shared 
 classes/structs, altough I didn't understand it too much (well 
 I didn't try too much to be honest).


Objects of a synchronized class automatically lock/unlock every 
method call.

You don't share stack structs, though maybe shared(StructType)*.


 For all those appoaches, which is the preffered?
 For all those appoaches, which is the preffered for game 
 programming?

 Thank you.


Hope I've helped.
NMS
Jun 27 2012
parent "Minas Mina" <minas_mina1990 hotmail.co.uk> writes:
You all helped, thank you :)

I will read the concurrency part tomorrow!
Jun 27 2012
prev sibling parent Dejan Lekic <dejan.lekic gmail.com> writes:
On Thu, 28 Jun 2012 00:34:50 +0200, Minas Mina wrote:


 I have been "playing" latetly with std.concurrency and core.thread. I
 like both, but they are entirely different approaches to concurrency.
 
 std.concurrency: Uses message passing. Does not allow passing of mutable
 types.
 core.thread: Essentially the same as the approach taken by Java.
 
 1) Is one favoured over the other? I guess std.concurrency would be,
 because it uses messages (and I recently read somewhere that
 immutability + message passing = good thing). Well, my problem about is,
 how can send mutable data to another thread? _Can_ I do it? If not, how
 am I supposed to share data between them? Not everything can be
 immutable right? For example I would like to have a thread calculate the
 sum of the first half of an array while another thread would calculate
 the other half, and I could add the two results at the end.
 
 2) What about core.thread? To do proper sync between threads in
 core.thread, semaphore, mutexes and stuff like that are needed. Are
 those "good practice" in D?
 
 3) I have also read a bit about syncronized classes and shared
 classes/structs, altough I didn't understand it too much (well I didn't
 try too much to be honest).
 
 For all those appoaches, which is the preffered?
 For all those appoaches, which is the preffered for game programming?
 
 Thank you.



Both std.concurrency and std.parallelism need core.thread in order to do 
what they do...


-- 
Dejan Lekic
  mailto:dejan.lekic(a)gmail.com
  http://dejan.lekic.org
Jun 29 2012