• Sean Kelly <sean invisibleduck.org> Aug 02 2010
• dsimcha <dsimcha yahoo.com> Aug 03 2010
• Sean Kelly <sean invisibleduck.org> Aug 03 2010
• dsimcha <dsimcha yahoo.com> Aug 03 2010
• bearophile <bearophileHUGS lycos.com> Aug 04 2010
• dsimcha <dsimcha yahoo.com> Aug 04 2010
• "Robert Jacques" <sandford jhu.edu> Aug 03 2010

Sean Kelly <sean invisibleduck.org> writes:

awishformore Wrote:
 
 I completely agree with everything you said and I really dislike how D2 
 currently seems to virtually impose an application architecture based on 
 the message passing model if you don't want to circumvent and thus break 
 the entire type system. While I do agree that message passing makes a 
 lot of sense as the default choice, there also has to be well 
 thought-out and extensive support for the shared memory model if D2 is 
 really focusing on the concurrency issue as much as it claims.
 
 Personally, I've found hybrid architectures where both models are 
 combined as needed to be the most flexible and best performing approach 
 and there is no way a language touted to be a systems language should 
 impose one model over the other and stop the programmer from doing 
 things the way he wants.


I wouldn't worry--the modules in core will always provide the low-level control
that you're looking for.  As for message passing, I think it should be the
default choice but not the only choice.  Things like futures are a reasonable
addition to Phobos as well.

dsimcha <dsimcha yahoo.com> writes:

== Quote from Sean Kelly (sean invisibleduck.org)'s article
 I wouldn't worry--the modules in core will always provide the low-level control


choice but not the only choice.  Things like futures are a reasonable addition
to
Phobos as well.

How explicit do we want to make inter-thread sharing when using a
futures/tasks/parallel foreach library?  This question may be a result of the
fact
that I (still) don't understand what shared is supposed to do, but do we want to
force all data to be shared/cast to shared even under a futures/tasks/parallel
foreach model, or do we want to bypass the shared system entirely and simply
stick
to old-fashioned "here be dragons" when doing future/task/parallel foreach-based
multithreading?

My personal opinion (which may change if I understand shared better) is that
"here
be dragons" is the way to go.  Futures/tasks/parallel foreach are meant for
parallelizing small hotspots, where the programmer generally knows implicitly
what
assumptions are being made, and generally revolve completely around shared
memory
and, where necessary, explicit locking.  Forcing things to be cast to shared
left,
right and center would simply be an annoying piece of syntactic salt that
doesn't
actually solve any problems in these cases.  (It would also completely destroy
the
nice, clean interface of the module I've created for task/future/parallel
foreach.)

I think Phobos's std.parallelism should just come with a big "here be dragons"
sign and that's about it as far as safety/use of shared/etc.

Sean Kelly <sean invisibleduck.org> writes:

dsimcha Wrote:

 == Quote from Sean Kelly (sean invisibleduck.org)'s article
 I wouldn't worry--the modules in core will always provide the low-level control


 choice but not the only choice.  Things like futures are a reasonable addition
to
 Phobos as well.
 
 How explicit do we want to make inter-thread sharing when using a
 futures/tasks/parallel foreach library?  This question may be a result of the
fact
 that I (still) don't understand what shared is supposed to do, but do we want
to
 force all data to be shared/cast to shared even under a futures/tasks/parallel
 foreach model, or do we want to bypass the shared system entirely and simply
stick
 to old-fashioned "here be dragons" when doing future/task/parallel
foreach-based
 multithreading?


Ideally, the parallelism should be both invisible to the user and automatically
safe.  That isn't a realistic goal though, so a balance must be found between
performance and safety.  I'd venture to say the "here be dragons" approach is
correct for the most part though.  If too much performance is traded away for
safety, no one will use it.

dsimcha <dsimcha yahoo.com> writes:

== Quote from Sean Kelly (sean invisibleduck.org)'s article
 dsimcha Wrote:
 == Quote from Sean Kelly (sean invisibleduck.org)'s article
 I wouldn't worry--the modules in core will always provide the low-level control


 choice but not the only choice.  Things like futures are a reasonable addition
to
 Phobos as well.

 How explicit do we want to make inter-thread sharing when using a
 futures/tasks/parallel foreach library?  This question may be a result of the
fact
 that I (still) don't understand what shared is supposed to do, but do we want
to
 force all data to be shared/cast to shared even under a futures/tasks/parallel
 foreach model, or do we want to bypass the shared system entirely and simply
stick
 to old-fashioned "here be dragons" when doing future/task/parallel
foreach-based
 multithreading?




performance and safety.  I'd venture to say the "here be dragons" approach is
correct for the most part though.  If too much performance is traded away for
safety, no one will use it.

My other major concern is that if too much convenience (in simple cases where
the
programmer does know what he/she is doing even if the guarantees aren't
statically
checkable) is traded for safety (i.e. by requiring stuff to be cast explicitly
to
shared in lots of places), the API will become almost unusable.

bearophile <bearophileHUGS lycos.com> writes:

Robert Jacques:
 Bartosz recently bloged about  
 task driven parallelism in three "High Productivity Computing Systems"  
 languages ( Chapel, X10, Fortress ) and criticized all three regarding  
 taking the "here be dragons" approach. Even experts wake up the dragons on  
 a semi-regular basis and debugging them back into slumber is both hard and  
 time consuming;


D has to learn a large number of things/tricks from Chapel. I have written two
posts to show some of them.

Bye,
bearophile

dsimcha <dsimcha yahoo.com> writes:

== Quote from Robert Jacques (sandford jhu.edu)'s article
 My experience with data-parallel programming leads me to believe that a
 large number of use cases could be covered by extending the D's set of
 function/member modifiers (i.e. const/shared/immutable/pure) to cover
 delegates. This would allow, for instance, a parallel foreach function to
 take a const delegate or a future function to take a shared delegate and
 thereby provide both safety and performance. Bartosz recently bloged about
 task driven parallelism in three "High Productivity Computing Systems"
 languages ( Chapel, X10, Fortress ) and criticized all three regarding
 taking the "here be dragons" approach.


Given that Bartosz is a type system guru I can see where he's coming from.
However, ironically we're talking about making such a library usable for mere
mortals and I consider myself a mere mortal when it comes the complexities of
type
systems, in that I find Bartosz's posts extremely theoretical and difficult to
follow.  I actually find it easier to visualize how work can be interleaved
between threads.

Since shared is relatively new and (I think) not fully implemented, immutable
is a
good example of why not everything can be easily expressed in the type system.
Immutable data has some wonderful theoretical properties, but creating immutable
data in D without either sacrificing a significant amount of efficiency (via
copying) or relying on unchecked casts, is close to impossible.  Yes, we could
have made unique a full-fledged type constructor, but that would have added
another level of complexity to the language when const/immutable already seems
complex to a lot of people.  The result of this is that much data that I share
across threads is logically immutable, but I've given up long ago on making most
cases of this statically checkable.

Also, with regard to the 10x development cost, I suspect a lot of that has to do
with getting the code to scale, too.  Code already becomes substantially harder
to
write, for example, when you can't  freely heap allocate whenever you want
(because you'd bottleneck on malloc and GC).  Since we're talking about shared
memory architectures here, I'll assume the cases you're referring to don't have
thread-local heaps and that at least some synchronization points are required
for
memory management.

"Robert Jacques" <sandford jhu.edu> writes:

On Tue, 03 Aug 2010 16:05:40 -0400, Sean Kelly <sean invisibleduck.org>  
wrote:

 dsimcha Wrote:

 == Quote from Sean Kelly (sean invisibleduck.org)'s article
 I wouldn't worry--the modules in core will always provide the  


 that you're looking for.  As for message passing, I think it should be  
 the default
 choice but not the only choice.  Things like futures are a reasonable  
 addition to
 Phobos as well.

 How explicit do we want to make inter-thread sharing when using a
 futures/tasks/parallel foreach library?  This question may be a result  
 of the fact
 that I (still) don't understand what shared is supposed to do, but do  
 we want to
 force all data to be shared/cast to shared even under a  
 futures/tasks/parallel
 foreach model, or do we want to bypass the shared system entirely and  
 simply stick
 to old-fashioned "here be dragons" when doing future/task/parallel  
 foreach-based
 multithreading?


 Ideally, the parallelism should be both invisible to the user and  
 automatically safe.  That isn't a realistic goal though, so a balance  
 must be found between performance and safety.  I'd venture to say the  
 "here be dragons" approach is correct for the most part though.  If too  
 much performance is traded away for safety, no one will use it.


My experience with data-parallel programming leads me to believe that a  
large number of use cases could be covered by extending the D's set of  
function/member modifiers (i.e. const/shared/immutable/pure) to cover  
delegates. This would allow, for instance, a parallel foreach function to  
take a const delegate or a future function to take a shared delegate and  
thereby provide both safety and performance. Bartosz recently bloged about  
task driven parallelism in three "High Productivity Computing Systems"  
languages ( Chapel, X10, Fortress ) and criticized all three regarding  
taking the "here be dragons" approach. Even experts wake up the dragons on  
a semi-regular basis and debugging them back into slumber is both hard and  
time consuming; it's typically to see a 10x cost in development time on  
massively parallel architectures. For mere mortals to be productive, those  
dragons need to be slain. And I think the D has a decent chance to be the  
first(?) non-functional language to slay them. But I think this should be  
a goal of D3, and that a "here be dragons" library for D2 would be  
extremely useful: because if we're going to slay the dragons, we need to  
know as about many caves and dragons as possible.