www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Concurrency in D?

reply "Kris" <fu bar.com> writes:
Here's just one article identifying what many of us already know ~ to more 
easily take advantage of the growing multicore trend, one needs tools with a 
bent towards such designs: http://theinq.net/?article=27247

Multicore is happening. Take a look here 
http://arstechnica.com/news.ars/post/20041106-4377.html, here 
http://www.realworldtech.com/page.cfm?ArticleID=RWT021005084318, here 
http://www.realworldtech.com/page.cfm?ArticleID=RWT102405055354 and possibly 
here http://arstechnica.com/articles/paedia/cpu/xbox360-2.ars/7.

We had a (very) brief discussion about 18 months ago as to what D might do 
to support such environments, and to perhaps leapfrog other contemporary 
languages in this regard. I say 'contemporary' since Occam 
http://en.wikipedia.org/wiki/Occam_programming_language was designed from 
the ground-up to support a model of distributed parallel computation (it 
failed to prosper due partly to a tight association with the Transputer)

What do you think D might do in this regard? Are there perhaps some notions 
from Occam (and/or Ada), beyond simple contention-controls, worth pursuing? 
Is there something at the language-level that might make notable differences 
vis-a-vis debugging parallel systems?

- Kris
Oct 26 2005
parent reply Sean Kelly <sean f4.ca> writes:
In article <djp91p$g2r$1 digitaldaemon.com>, Kris says...

We had a (very) brief discussion about 18 months ago as to what D might do 
to support such environments, and to perhaps leapfrog other contemporary 
languages in this regard. I say 'contemporary' since Occam 
http://en.wikipedia.org/wiki/Occam_programming_language was designed from 
the ground-up to support a model of distributed parallel computation (it 
failed to prosper due partly to a tight association with the Transputer)

What do you think D might do in this regard? Are there perhaps some notions 
from Occam (and/or Ada), beyond simple contention-controls, worth pursuing? 
Is there something at the language-level that might make notable differences 
vis-a-vis debugging parallel systems?


There was a rather lengthy thread on this in comp.arch a few months ago:



Cilk may be worth looking at as well, as it's a parallel version of C:

http://supertech.lcs.mit.edu/cilk/

I think D is already very well suited for common parallel programming techniques
as it supports inline assembler, language-level threads, the volatile statement,
and it has garbage collection.  These features allow for safe, language
complaint two phase update, lock free, and other populalr techniques... and
things will be even better once we have stack-based classes.

Beyond that, I'll have to give it some thought.  I think Cilk may be a good
reference for desirable features, but I haven't gotten further with it than
skimming the docs :-)


Sean
Oct 26 2005
next sibling parent reply dennis luehring <dl.soluz gmx.net> writes:
 Cilk may be worth looking at as well, as it's a parallel version of C:
 
 http://supertech.lcs.mit.edu/cilk/


and corn: http://corn.telefonia.pl/

i like the


   func1(); // thread1
   func2(); // thread2
}

syntax very much
Oct 27 2005
parent Sean Kelly <sean f4.ca> writes:
In article <djrl39$1l0m$1 digitaldaemon.com>, dennis luehring says...

 Cilk may be worth looking at as well, as it's a parallel version of C:
 
 http://supertech.lcs.mit.edu/cilk/


and corn: http://corn.telefonia.pl/

i like the


   func1(); // thread1
   func2(); // thread2
}

syntax very much


Ah, that's like "parbegin" and "parend" that I was introduced to in an OS design
class, though IIRC those were for process-level concurrency--they certainly beat
the heck out of fork() though :-)


Sean
Oct 27 2005
prev sibling parent reply "Kris" <fu bar.com> writes:
Thanks for the links, Sean. That forum had some interesting comments ~ 
especially regarding transactable execution.

D certainly does has a good range of primitives for thread-support, though 
some might conceivably be exposed in a somewhat simpler (and perhaps 
smarter) fashion than they currently are. But, that aside, I was kinda' 
thinking more along the lines of debugging assistance ~ a known bugbear of 
multithreading development.

For example, how about the compiler supporting some kind of read/write 
coloring, to indicate potential read/write conflicts between threads? That 
would be incredibly useful, and not hard for the compiler to identify, 
assuming it had some kind of PAR support to recognize threaded sections ... 
that is ~ once the compiler has some basic knowledge of what the programmer 
is attempting, it can provide seriously powerful diagnostics.

I suspect there's relatively few of us who have notable experience with 
multi-threading, as this article asserts (MS architect says "developers 
ill-prepared for new CPUs") 
http://arstechnica.com/news.ars/post/20051027-5492.html.

Is that the case? How about an informal poll of the D community:

===============

1) Do you have reasonable experience with the problems associated with MT, 
and feel reasonably comfortable with that development model?

2) Assuming it were financially attractive, could you make what you feel to 
be educated & valid assertions about how to successfully migrate a 
'traditional' application to take full advantage of the new breed of CPU's?







"Sean Kelly" <sean f4.ca> wrote in message 
news:djppmg$12om$1 digitaldaemon.com...

 In article <djp91p$g2r$1 digitaldaemon.com>, Kris says...

We had a (very) brief discussion about 18 months ago as to what D might do
to support such environments, and to perhaps leapfrog other contemporary
languages in this regard. I say 'contemporary' since Occam
http://en.wikipedia.org/wiki/Occam_programming_language was designed from
the ground-up to support a model of distributed parallel computation (it
failed to prosper due partly to a tight association with the Transputer)

What do you think D might do in this regard? Are there perhaps some 
notions
from Occam (and/or Ada), beyond simple contention-controls, worth 
pursuing?
Is there something at the language-level that might make notable 
differences
vis-a-vis debugging parallel systems?


 There was a rather lengthy thread on this in comp.arch a few months ago:



 Cilk may be worth looking at as well, as it's a parallel version of C:

 http://supertech.lcs.mit.edu/cilk/

 I think D is already very well suited for common parallel programming 
 techniques
 as it supports inline assembler, language-level threads, the volatile 
 statement,
 and it has garbage collection.  These features allow for safe, language
 complaint two phase update, lock free, and other populalr techniques... 
 and
 things will be even better once we have stack-based classes.

 Beyond that, I'll have to give it some thought.  I think Cilk may be a 
 good
 reference for desirable features, but I haven't gotten further with it 
 than
 skimming the docs :-)


 Sean
Oct 30 2005
next sibling parent "Regan Heath" <regan netwin.co.nz> writes:
On Sun, 30 Oct 2005 13:57:01 -0800, Kris <fu bar.com> wrote:

 1) Do you have reasonable experience with the problems associated with  
 MT, and feel reasonably comfortable with that development model?


Yes. 4+ years developing a multi-threaded email server for windows, linux,  
freebsd, solaris (x86 and sparc), mac osx, etc. 7+ years in that area  
(web/email server) of development i.e. (single-threaded) web cgi support  
products and modules for use with the email server. 2+ years doing GUI  
style work (inherently multi-threaded), also support products for the  
email server.


 2) Assuming it were financially attractive, could you make what you feel  
 to be educated & valid assertions about how to successfully migrate a
 'traditional' application to take full advantage of the new breed of  
 CPU's?


I believe so.





My boss and 4 or 5 (ex-)workmates.

Regan



 "Sean Kelly" <sean f4.ca> wrote in message
 news:djppmg$12om$1 digitaldaemon.com...

 In article <djp91p$g2r$1 digitaldaemon.com>, Kris says...

 We had a (very) brief discussion about 18 months ago as to what D  
 might do
 to support such environments, and to perhaps leapfrog other  
 contemporary
 languages in this regard. I say 'contemporary' since Occam
 http://en.wikipedia.org/wiki/Occam_programming_language was designed  
 from
 the ground-up to support a model of distributed parallel computation  
 (it
 failed to prosper due partly to a tight association with the  
 Transputer)

 What do you think D might do in this regard? Are there perhaps some
 notions
 from Occam (and/or Ada), beyond simple contention-controls, worth
 pursuing?
 Is there something at the language-level that might make notable
 differences
 vis-a-vis debugging parallel systems?


 There was a rather lengthy thread on this in comp.arch a few months ago:



 Cilk may be worth looking at as well, as it's a parallel version of C:

 http://supertech.lcs.mit.edu/cilk/

 I think D is already very well suited for common parallel programming
 techniques
 as it supports inline assembler, language-level threads, the volatile
 statement,
 and it has garbage collection.  These features allow for safe, language
 complaint two phase update, lock free, and other populalr techniques...
 and
 things will be even better once we have stack-based classes.

 Beyond that, I'll have to give it some thought.  I think Cilk may be a
 good
 reference for desirable features, but I haven't gotten further with it
 than
 skimming the docs :-)


 Sean
Oct 30 2005
prev sibling next sibling parent reply Georg Wrede <georg.wrede nospam.org> writes:
(PS: the tone here might appear aggressive, but wasn't meant as such.)

Kris wrote:

 1) Do you have reasonable experience with the problems associated 
 with MT, and feel reasonably comfortable with that development model?
 


Yes. Apart from critical sections, race conditions, and such, MT really
is not hard. As long as you manage to stay on top of what you yourself
are doing. Case in point: somewhere in the '90s, I organized a seminar
for top-level Java programmers, in Lapland. In my spare time, I attended
some of the more interesting lectures.

One of the points I never will forget was, that an object instance, with 
its current state, may have several simultaneous threads executing its 
code -- within its current state.

I didn't sleep the next night. But ever since, I've felt comfortable
with MT.


 2) Assuming it were financially attractive, could you make what you 
 feel to be educated & valid assertions about how to successfully 
 migrate a 'traditional' application to take full advantage of the new
  breed of CPU's?


No. But that's mainly because I have a hard time understanding what one
vs. two (or more) processors have to do with this.

On the conceptual level, that is. Of course (if this were a job
interview) I could start about spreading the work load between a "ui
process" and a "controller process" or some such BS.

But from a programmer's point of view, you either have MT or you don't.
D has it right in the language, and even if it weren't, you could rather
easily implement cooperative MT yourself, in D. Of course, this isn't
preemptive multitasking, but that was not asked for.

Then again, preemptive multitasking, as presented by the operating
system, is heavy compared with (self-made) cooperative multitasking.
Therefore, they should be used in dissimilar contexts.

---

As to "migrate a 'traditional' application to take full advantage of the
new breed of CPUs", I'd like to be as arrogant as to say, the issue is
way broader than can be asked in one sentence. It's not like there's a
known (or even unknown) receipe on how to do it. The 'traditional'
applications are diverse enough to preclude any single method of
conversion.

(Examples: A traditional app to count vehicles on the Bay Bridge, in
real time, getting input only from an under-road inductance loop, and
differentiating between regular cars and utility vehicles. A bank
back-room application monitoring input and output of cashiers' bought
and sold notes in a currency exchange office. Any old mainframe app
designed to talk with hundreds of VT-100 terminals, converted to be used
with Internet browsers instead. (Or was it a new in-between server,
serving the browsers and appearing to the mainframe as hundreds of VT
terminals?))






asking, my answer would be 3-6 people I know. But then, they aren't
exactly average programmers either.

----

So where does this leave us?

I think you might rephrase the question. First of all, to more specific,
and secondly, more to what you're after. For example, early in your
post, you mention "transactable execution". This naturally biases the
answers towards database related thinking -- right?

Which again raises the question: are we charting things that should be
in the language, or are we charting what should "sort of" be included in
"the eventual standard distribution of D"?
Oct 30 2005
next sibling parent Sean Kelly <sean f4.ca> writes:
Georg Wrede wrote:

 (PS: the tone here might appear aggressive, but wasn't meant as such.)
 
 Kris wrote:
 

 1) Do you have reasonable experience with the problems associated with 
 MT, and feel reasonably comfortable with that development model?


 Yes. Apart from critical sections, race conditions, and such, MT really
 is not hard. As long as you manage to stay on top of what you yourself
 are doing.


I'd add deadlock resistance to the list of difficult MT problems.  This 
by itself is one of the strongest arguments for lock-free concurrency IMO.


 2) Assuming it were financially attractive, could you make what you 
 feel to be educated & valid assertions about how to successfully 
 migrate a 'traditional' application to take full advantage of the new
  breed of CPU's?


 
 No. But that's mainly because I have a hard time understanding what one
 vs. two (or more) processors have to do with this.


The hardware layer doesn't matter as such.  The primary reason this is a 
concern with respect to new CPUs is that clock speed is beginning to 
plateau, so performance scaling is moving away from clock speed towards 
parallelization.  This has been a growing concern among the C++ and MT 
communities for a while now.


 But from a programmer's point of view, you either have MT or you don't.
 D has it right in the language, and even if it weren't, you could rather
 easily implement cooperative MT yourself, in D. Of course, this isn't
 preemptive multitasking, but that was not asked for.


I agree with Kris that it's worth considering other language features to 
support or facilitate concurrent programming.  While concurrency is a 
fairly well understood problem by some, the support for it in popular 
procedural languages is pretty weak.  Java is better off than most, and 
it doesn't even go much further than "synchronized" and a memory model 
definition.  D may do quite well to have some additional concurrent 
programming features available (similar to those in Cilk, for example).


 As to "migrate a 'traditional' application to take full advantage of the
 new breed of CPUs", I'd like to be as arrogant as to say, the issue is
 way broader than can be asked in one sentence. It's not like there's a
 known (or even unknown) receipe on how to do it. The 'traditional'
 applications are diverse enough to preclude any single method of
 conversion.


It would be nice if there were a fairly systematic way for doing this, 
but I agree that there may not be.  The Intel compiler, for example, can 
parallelize some code blocks automatically, but this amounts to a very 
small portion of the code (only loops IIRC).


 Which again raises the question: are we charting things that should be
 in the language, or are we charting what should "sort of" be included in
 "the eventual standard distribution of D"?


A bit of both, I think.


Sean
Oct 30 2005
prev sibling parent "Kris" <fu bar.com> writes:
Good points. Some comments inline:

"Georg Wrede" <georg.wrede nospam.org> ...

 (PS: the tone here might appear aggressive, but wasn't meant as such.)

 Kris wrote:

 1) Do you have reasonable experience with the problems associated with 
 MT, and feel reasonably comfortable with that development model?


 Yes. Apart from critical sections, race conditions, and such, MT really
 is not hard. As long as you manage to stay on top of what you yourself
 are doing. <snip>


That's a large part of what the linked article was getting at. If a compiler 
(or other tool) could help more in this regard it could be a big win. The 
Concur project at MS is apparently hoping to do just that. The 
write-conflict coloring I mentioned earlier would be an example of such a 
win: it would identify all possible read/write & write/write conflicts, 
thereby supporting a developer in their quest to "stay on top of things" (as 
you put it so well).



 2) Assuming it were financially attractive, could you make what you feel 
 to be educated & valid assertions about how to successfully migrate a 
 'traditional' application to take full advantage of the new
  breed of CPU's?


 No. But that's mainly because I have a hard time understanding what one
 vs. two (or more) processors have to do with this.


Only that to take full advantage of the lastest CPUs, one likely has to 
become familiar with MT. Whereas before, that was rarely a concern for a 
large swathe of application types (and programmers).

<snip>

 As to "migrate a 'traditional' application to take full advantage of the
 new breed of CPUs", I'd like to be as arrogant as to say, the issue is
 way broader than can be asked in one sentence. It's not like there's a
 known (or even unknown) receipe on how to do it. The 'traditional'
 applications are diverse enough to preclude any single method of
 conversion.


Yes; you are quite right. I should have been more specific, and narrow the 
scope to those applications that stood to gain most from an ever-increasing 
single-core CPU throughput (MHz is one vector). Such apps rarely deal with 
MT issues directly (except for the occasional "execute later" API present in 
certain GUIs). Now that the performance gains are slanted more towards 
multi-core, developers will perhaps /have/ to face the MT hurdle. Exactly 
what that linked article was getting at.

On the other hand, I wanted to leave the question open to interpretation. 
There's many ways to skin the MT cat, as you note; but only those with prior 
experience would likely be familiar with the concern vectors.






 asking, my answer would be 3-6 people I know. But then, they aren't
 exactly average programmers either.


I think that last sentence identifies an important point. If D could help 
those 'average' programmers succeed by, for example, pointing out in the 
code where there might be trouble, then it would stand head-and-shoulders 
above any other language in this respect (and I'm guessing that this kind of 
thing is an aspect of the Concur project?).

<snip>

 I think you might rephrase the question. First of all, to more specific,
 and secondly, more to what you're after. For example, early in your
 post, you mention "transactable execution". This naturally biases the
 answers towards database related thinking -- right?


By "transactable execution" I was reflecting the ability to roll-back the 
effect of execution ~ any execution ~ within certain contraints. There's 
been a lot of thought in that area over the last 30 years (or whatever) 
since it would allow parallel /speculative/ execution at a much 
higher/large-grained level than is currently possible at the die/branch 
level. Traditionally, that kind of thing has been built into DBs to gain 
ACID properties (atomic, consistent, isolated, durable), but it has other 
advantages too. Still, that's all rather speculative at this point, and 
doesn't really have much bearing on the diagnostic type of tools I'd like to 
see. I mentioned it only as reference to the links posted by Sean.


 Which again raises the question: are we charting things that should be
 in the language, or are we charting what should "sort of" be included in
 "the eventual standard distribution of D"?


Specifically, I was hoping to concentrate on what language attributes (as 
opposed to library attributes) could help developers in the MT world. Being 
able to extract features from the code (such as parallel execution) can lead 
to the coloring example I mentioned. Again, this is just speculation.

I have several other half-baked notions in mind, but they don't really 
belong in this "poll" thread so I'll just briefly mention a couple of them:

a) A little more than the simple (yet powerful) 'synchronized' keyword; 
perhaps some of the CS constructs that Doug Lea implements & documents so 
well? Having some related extractable features at the language level would 
likely aid diagnostics in a similar manner to a PAR type construct, since 
they tend to go hand-in-hand.

b) A multi-instance GC. Single-pool allocators, and a GC in particular, can 
be a serious bottleneck for multi-threaded apps. More so than 
single-threaded since there's now contention and stalling in the allocator, 
and often severe contention in the collector (i.e. halts all running 
threads). I tend to avoid allocations at all cost within a server; precisely 
for this reason (e.g. the Mango HTTP and Servlet servers do not allocate any 
per-request memory, once they have warmed up).

Thanks for the feedback, Georg.
Oct 30 2005
prev sibling parent reply Sean Kelly <sean f4.ca> writes:
Kris wrote:

 Thanks for the links, Sean. That forum had some interesting comments ~ 
 especially regarding transactable execution.


I think so too.  For anyone interested, the post that made me think of 
the thread in the first place is here (154. Andy Glew):



It does a rather good job of summing up most of the issues.  Now on to 
the poll!


 1) Do you have reasonable experience with the problems associated with MT, 
 and feel reasonably comfortable with that development model?


Yes.


 2) Assuming it were financially attractive, could you make what you feel to 
 be educated & valid assertions about how to successfully migrate a 
 'traditional' application to take full advantage of the new breed of CPU's?


I believe so, though I'd have to think about whether I could suggest a 
mechanical approach for doing so without being overly general.





Of my current coworkers, I can think of of one person.  I haven't worked 
with a terribly large number of people who have much MT programming 
experience or who know architecture issues well enough to compensate.


Sean
Oct 31 2005
parent Georg Wrede <georg.wrede nospam.org> writes:
I took a peek at Cilk. The academic research behind it seems quite 
solid, and the additions to C are kept to a minimum.

I wonder, has any of you guys looked at the preprocessor? Would it be 
hard to convert it to work on D code?
Oct 31 2005