www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Rob Pike's Newsqueak - some good concepts

reply Dejan Lekic <dejan.lekic gmail.com> writes:
Today I watched an excellent presentation by Mr. Rob Pike on Google Tech Talks.
(Advanced Topics in Programming Languages: Concurrency/message passing
Newsqueak - http://video.google.com/url?docid=810232012617965344&esrc=rss_uds&ev=v&q=user:%22Google+engEDU%22&vidurl=http://video.google.com/videoplay%3Fdocid%3D810232012617965344%26q%3Duser%253A%2522Google%2BengEDU%2522%26hl%3Den&usg=AL29H22XDZMMUf
udHDB2dqX9jtFEE4L9w )
There are several very interesting concepts in his Newsqueak langauge which I
would be very happy to see in D. One of such is the ability to execute ANY
function as a new process (thread) via keyword "begin".
An example:
<code>
// makes no sense, but should work
begin int sum(int a, int b) { return a+b; } 

begin int doConcurrently(int a, int b)
{
  for (; a>0; a--)
    print(b);  // prints in background
}

// same, like lambda
begin int doConcurrently2(int a, int b)
{
  for (; a>0; a--)
    print(b);  // prints in background
}(20, sum(23, 5))
</code>

As You can see, begin makes expression/function work in a separate
process/thread. It has been discussed about having lambda-functions here on
these newsgroups, so I will not touch that topic in this thread...

Second very interesting concept is connected with the "become" keyword. With it
you can replace function with it's execution, which basically generalizes tail
recursion. This is better explained with an example:
<code>
// Any expression
int function(int a, int b) { become a + b; }

int sum(int a, int b)
{
  return a+b;
}

// Any function invocation
// Reuses sum's stack space!
int difference(int a, int b) 
{
  become sum(a, -b);
}
</code>

There are more interesting things, I haven't had time to write about them, so I
strongly recommend seeing this presentation! :)

Cheers!
May 18 2007
next sibling parent Dejan Lekic <dejan.lekic gmail.com> writes:
Yes, I have forgotten to say something about chan-nels - and the "select"
statement which is part of the languge. I would BEG Mr. Walter to incorporate
something like this into D! :)
May 18 2007
prev sibling next sibling parent reply Mikola Lysenko <mclysenk mtu.edu> writes:
Concurrent programming has enormous applications, but it has yet to gain much
mainstream traction yet.  Partly because present implementations are
inefficient, and partly because there is a substantial knowledge barrier to
entry.

This is a shame, since you can easily write all sorts of programs extremely
elegantly.  Especially interactive applications and simulations, such as games,
webapps, databases, fluid dynamics code etc.

Right now, it is possible to write CSP-style programs in D.  DCSP already does
all of these things discussed in this talk, though the structure has more in
common with occam than Newsqueak.  You can check it out at:

http://assertfalse.com

Part of the problem with CSP-style languages is that implementing a
non-deterministic choice operator or 'guarded' select is that it is very
difficult.  Frankly, the version in DCSP is rather slow and unnecessarily
limited.  In the future, I hope to address these issues, but I won't be able to
do any work until I finish up my current projects.

-Mik
May 18 2007
next sibling parent Dejan Lekic <dejan.lekic gmail.com> writes:
Mr. Lysenko, thanks for the information! I have a link to assert(false) from my
home page since last year, but I haven't seen DCSP! Excellent! :)
May 18 2007
prev sibling next sibling parent reply noname <noname lolrofl.fakeaddress.net> writes:
Mikola Lysenko Wrote:


 Concurrent programming has enormous applications, but it has yet to gain much
mainstream traction yet.  Partly because present implementations are
inefficient, and partly because there is a substantial knowledge barrier to
entry.
 
 
 -Mik


It is surprising how much difference in performance there is between
implementations of concurrency (mutexes and threads) between different
languages. 

If I refer to this:
http://shootout.alioth.debian.org/gp4/benchmark.php?test=chameneos&lang=all
http://shootout.alioth.debian.org/gp4/benchmark.php?test=message&lang=all

Haskell seems to perform the best, while Java and C++ (pthreads) are mediocre.
D seems to be doing not too bad.
May 19 2007
next sibling parent renoX <renosky free.fr> writes:
noname a crit :

 Mikola Lysenko Wrote:
 

 Concurrent programming has enormous applications, but it has yet to gain much
mainstream traction yet.  Partly because present implementations are
inefficient, and partly because there is a substantial knowledge barrier to
entry.


 -Mik


 
 It is surprising how much difference in performance there is between
implementations of concurrency (mutexes and threads) between different
languages. 
 
 If I refer to this:
 http://shootout.alioth.debian.org/gp4/benchmark.php?test=chameneos&lang=all
 http://shootout.alioth.debian.org/gp4/benchmark.php?test=message&lang=all
 
 Haskell seems to perform the best, while Java and C++ (pthreads) are mediocre.
D seems to be doing not too bad.


Note that this kind of mono-CPU benchmark without knowledge of how it 
scales to SMP CPUs is not very useful IMHO.

renoX
May 19 2007
prev sibling parent eao197 <eao197 intervale.ru> writes:
On Sat, 19 May 2007 13:32:02 +0400, noname  =

<noname lolrofl.fakeaddress.net> wrote:


 It is surprising how much difference in performance there is between  =



 implementations of concurrency (mutexes and threads) between different=


  =


 languages.

 If I refer to this:
 http://shootout.alioth.debian.org/gp4/benchmark.php?test=3Dchameneos&l=


ang=3Dall

 http://shootout.alioth.debian.org/gp4/benchmark.php?test=3Dmessage&lan=


g=3Dall

 Haskell seems to perform the best, while Java and C++ (pthreads) are  =



 mediocre. D seems to be doing not too bad.


Some languages (like Erlang and Haskell) don't use native thread. They u=
se  =

their own threading models (like green threads) which much more efficien=
t  =

in such tests. But green threads could block all application if some  =

thread is doing some long time blocking operation (like blocking I/O).

And some implementation of tests in The Great Language Shootout have  =

serious bugs so don't trust those results too much.

-- =

Regards,
Yauheni Akhotnikau
May 19 2007
prev sibling parent "Joel C. Salomon" <JoelCSalomon Gmail.com> writes:
Mikola Lysenko wrote:

 Part of the problem with CSP-style languages is that implementing a
non-deterministic choice operator or 'guarded' select is that it is very
difficult.  Frankly, the version in DCSP is rather slow and unnecessarily
limited.  In the future, I hope to address these issues, but I won't be able to
do any work until I finish up my current projects.


The Newsqueak papers talk about the implementation of “select”; the Plan 
9 thread(2) or Russ Cox’s libtask have the source code for the “Alt” 
structure and its related code.  It’s a solved problem.

--Joel
May 20 2007
prev sibling next sibling parent Daniel Keep <daniel.keep.lists gmail.com> writes:
Dejan Lekic wrote:

 Today I watched an excellent presentation by Mr. Rob Pike on Google Tech
Talks. (Advanced Topics in Programming Languages: Concurrency/message passing
Newsqueak - http://video.google.com/url?docid=810232012617965344&esrc=rss_uds&ev=v&q=user:%22Google+engEDU%22&vidurl=http://video.google.com/videoplay%3Fdocid%3D810232012617965344%26q%3Duser%253A%2522Google%2BengEDU%2522%26hl%3Den&usg=AL29H22XDZMMUf
udHDB2dqX9jtFEE4L9w )
 There are several very interesting concepts in his Newsqueak langauge which I
would be very happy to see in D. One of such is the ability to execute ANY
function as a new process (thread) via keyword "begin".
 An example:
 <code>
 // makes no sense, but should work
 begin int sum(int a, int b) { return a+b; } 
 
 begin int doConcurrently(int a, int b)
 {
   for (; a>0; a--)
     print(b);  // prints in background
 }
 
 // same, like lambda
 begin int doConcurrently2(int a, int b)
 {
   for (; a>0; a--)
     print(b);  // prints in background
 }(20, sum(23, 5))
 </code>
 
 As You can see, begin makes expression/function work in a separate
process/thread. It has been discussed about having lambda-functions here on
these newsgroups, so I will not touch that topic in this thread...
 
 Second very interesting concept is connected with the "become" keyword. With
it you can replace function with it's execution, which basically generalizes
tail recursion. This is better explained with an example:
 <code>
 // Any expression
 int function(int a, int b) { become a + b; }
 
 int sum(int a, int b)
 {
   return a+b;
 }
 
 // Any function invocation
 // Reuses sum's stack space!
 int difference(int a, int b) 
 {
   become sum(a, -b);
 }
 </code>
 
 There are more interesting things, I haven't had time to write about them, so
I strongly recommend seeing this presentation! :)
 
 Cheers!


Both are pretty cool, but the first sounds a lot like the future
function myself and someone else wrote a while back.  Forgive me if I'm
misunderstanding, but it basically moves "begin" to the point of use:


 int sum(int a, int b) { return a+b; }

 // ...

 auto result = future(sum, 38, 4);
 // Do something else
 assert( result.value == 42 ); // .value blocks until other thread has
                               // computed the result


I'm not sure where mine got to, but I'm pretty sure the other one is in
scrapple on dsource.org/projects/scrapple.

	-- Daniel

-- 
int getRandomNumber()
{
    return 4; // chosen by fair dice roll.
              // guaranteed to be random.
}

http://xkcd.com/

v2sw5+8Yhw5ln4+5pr6OFPma8u6+7Lw4Tm6+7l6+7D
i28a2Xs3MSr2e4/6+7t4TNSMb6HTOp5en5g6RAHCP  http://hackerkey.com/
May 18 2007
prev sibling next sibling parent reply renoX <renosky free.fr> writes:
Dejan Lekic a crit :

 Today I watched an excellent presentation by Mr. Rob Pike on Google
 Tech Talks. (Advanced Topics in Programming Languages:
 Concurrency/message passing Newsqueak -
 http://video.google.com/url?docid=810232012617965344&esrc=rss_uds&ev=v&q=user:%22Google+engEDU%22&vidurl=http://video.google.com/videoplay%3Fdocid%3D810232012617965344%26q%3Duser%253A%2522Google%2BengEDU%2522%26hl%3Den&usg=AL29H22XDZMMUfZudHDB2dqX9jtFEE4L9w
 ) There are several very interesting concepts in his Newsqueak


Ah, limbo/newsqueak, such a nice syntax..
In many ways, it's much better than D's syntax IMHO.

Of course when you pass everything by copy, the syntax is nicer (no 
const, final,.. mess), but the price to pay in performance must be 
awful; still it's probably the right choice to make for a scripting 
language.


That said I thought that the content of the presentation is a bit 'old': 
after all he was presenting 20+ old ideas.

renoX





 langauge which I would be very happy to see in D. One of such is the
 ability to execute ANY function as a new process (thread) via keyword
 "begin". An example: <code> // makes no sense, but should work begin
 int sum(int a, int b) { return a+b; }
 
 begin int doConcurrently(int a, int b) { for (; a>0; a--) print(b);
 // prints in background }
 
 // same, like lambda begin int doConcurrently2(int a, int b) { for (;
 a>0; a--) print(b);  // prints in background }(20, sum(23, 5)) 
 </code>
 
 As You can see, begin makes expression/function work in a separate
 process/thread. It has been discussed about having lambda-functions
 here on these newsgroups, so I will not touch that topic in this
 thread...
 
 Second very interesting concept is connected with the "become"
 keyword. With it you can replace function with it's execution, which
 basically generalizes tail recursion. This is better explained with
 an example: <code> // Any expression int function(int a, int b) {
 become a + b; }
 
 int sum(int a, int b) { return a+b; }
 
 // Any function invocation // Reuses sum's stack space! int
 difference(int a, int b) { become sum(a, -b); } </code>
 
 There are more interesting things, I haven't had time to write about
 them, so I strongly recommend seeing this presentation! :)
 
 Cheers!
May 18 2007
next sibling parent "Joel C. Salomon" <JoelCSalomon Gmail.com> writes:
renoX wrote:

 That said I thought that the content of the presentation is a bit 'old': 
 after all he was presenting 20+ old ideas.


Twenty-year-old ideas that /still/ haven’t gotten their fair use except 
in one obscure (but wicked cool!) OS and one fairly weird and still 
obscure embedded processor—anyone do any Transputer work recently?

Read Pike’s paper on the implementations of 8½ & rio; these are powerful 
ideas that programmers deserve to have available to them.

--Joel
May 20 2007
prev sibling parent maht <maht-digitalmars.com mail.maht0x0r.net> writes:
Limbo has a ref keyword that makes a ref to an adt so you just copy refs around
not huge copies of the data structure.

I've done a postgresql client in Limbo, it's great having the CSP co-routines
just
chat away to each other. It really separates out the way you approach a problem.
May 22 2007
prev sibling parent "Joel C. Salomon" <JoelCSalomon Gmail.com> writes:
Dejan Lekic wrote:

 Today I watched an excellent presentation by Mr. Rob Pike on Google Tech
Talks. (Advanced Topics in Programming Languages: Concurrency/message passing
Newsqueak - http://video.google.com/url?docid=810232012617965344&esrc=rss_uds&ev=v&q=user:%22Google+engEDU%22&vidurl=http://video.google.com/videoplay%3Fdocid%3D810232012617965344%26q%3Duser%253A%2522Google%2BengEDU%2522%26hl%3Den&usg=AL29H22XDZMMUf
udHDB2dqX9jtFEE4L9w )
 There are several very interesting concepts in his Newsqueak langauge which I
would be very happy to see in D. One of such is the ability to execute ANY
function as a new process (thread) via keyword "begin".


I mentioned some of these ideas when the discussion of a threading 
library for D came up in discussion.  Remember, though, that “procs” 
(what Pike calls his light-weight processes) as implemented in Newsqueak 
are maybe not suited for a systems programming language like D.  On the 
other hand, Alef /was/ a systems language.

See the Alef papers at <http://plan9.bell-labs.com/wiki/plan9/papers/>, 
the Plan 9 thread(2) man page for the C port of these concepts at 
<http://plan9.bell-labs.com/magic/man2html/2/thread>, and Russ Cox’s 
libtask at <http://swtch.com/libtask/>, a stripped-down version of the 
library ported to Unix.

I’ve played with some of the Plan 9 programs built on Pike’s concurrency 
concept.  I think it’s a powerful concept, and well-suited to D. 
Whether D needs built-in keywords for this (the Newsqueak/Limbo/Alef 
model) or a library (like thread(2) or libtask, but with a D-ish 
interface) I wouldn’t presume to say.

--Joel
May 20 2007