www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - MPI Concurrency library update?

reply "Andrew Spott" <andrew.spott gmail.com> writes:
A while ago, (August of last year I think), there was talk about
a MPI wrapper for D.  Has there been any update on that?
Aug 10 2012
parent reply "dsimcha" <dsimcha yahoo.com> writes:
On Friday, 10 August 2012 at 23:40:43 UTC, Andrew Spott wrote:

 A while ago, (August of last year I think), there was talk about
 a MPI wrapper for D.  Has there been any update on that?


I was considering writing one, but I wanted it to be high-level 
and easy-to-use.  I ended up not doing it, initially because I 
was waiting for serialization to be added to Phobos (which I 
thought was imminent) and then because I got busy with unrelated 
things.

I think that a nice high-level MPI wrapper for D should be 
tightly integrated into a serialization library to encapsulate 
the low-level details of passing non-trivial data structures 
across nodes.  I doubt I'll get around to implementing it when 
serialization is added, though, because I'm probably past the 
MPI-using stage of my life (my Ph.D. research is basically 
finished, I'm just revising my dissertation and preparing to 
defend) so I wouldn't get to eat my own dogfood.
Aug 10 2012
next sibling parent reply "Andrew" <andrew.spott gmail.com> writes:
On Saturday, 11 August 2012 at 00:24:40 UTC, dsimcha wrote:

 On Friday, 10 August 2012 at 23:40:43 UTC, Andrew Spott wrote:

 A while ago, (August of last year I think), there was talk 
 about
 a MPI wrapper for D.  Has there been any update on that?


 I was considering writing one, but I wanted it to be high-level 
 and easy-to-use.  I ended up not doing it, initially because I 
 was waiting for serialization to be added to Phobos (which I 
 thought was imminent) and then because I got busy with 
 unrelated things.

 I think that a nice high-level MPI wrapper for D should be 
 tightly integrated into a serialization library to encapsulate 
 the low-level details of passing non-trivial data structures 
 across nodes.  I doubt I'll get around to implementing it when 
 serialization is added, though, because I'm probably past the 
 MPI-using stage of my life (my Ph.D. research is basically 
 finished, I'm just revising my dissertation and preparing to 
 defend) so I wouldn't get to eat my own dogfood.


Well, my PhD research is just beginning... :)

Any chance you could pass on what you have?  It might help me out
a bit, and reduce my workload toward creating a usable MPI
library.

Thanks.

-Andrew
Aug 10 2012
next sibling parent "dsimcha" <dsimcha yahoo.com> writes:
All I have is a very ad-hoc wrapper that does just what I needed 
for my purposes.  It basically has function prototypes for the 
parts of the API I actually care about and a few high-level 
wrappers for passing primitives and arrays to other nodes of the 
same architecture.

On Saturday, 11 August 2012 at 01:12:29 UTC, Andrew wrote:

 On Saturday, 11 August 2012 at 00:24:40 UTC, dsimcha wrote:

 On Friday, 10 August 2012 at 23:40:43 UTC, Andrew Spott wrote:

 A while ago, (August of last year I think), there was talk 
 about
 a MPI wrapper for D.  Has there been any update on that?


 I was considering writing one, but I wanted it to be 
 high-level and easy-to-use.  I ended up not doing it, 
 initially because I was waiting for serialization to be added 
 to Phobos (which I thought was imminent) and then because I 
 got busy with unrelated things.

 I think that a nice high-level MPI wrapper for D should be 
 tightly integrated into a serialization library to encapsulate 
 the low-level details of passing non-trivial data structures 
 across nodes.  I doubt I'll get around to implementing it when 
 serialization is added, though, because I'm probably past the 
 MPI-using stage of my life (my Ph.D. research is basically 
 finished, I'm just revising my dissertation and preparing to 
 defend) so I wouldn't get to eat my own dogfood.


 Well, my PhD research is just beginning... :)

 Any chance you could pass on what you have?  It might help me 
 out
 a bit, and reduce my workload toward creating a usable MPI
 library.

 Thanks.

 -Andrew
Aug 11 2012
prev sibling parent reply Russel Winder <russel winder.org.uk> writes:
On Sat, 2012-08-11 at 03:12 +0200, Andrew wrote:

 On Saturday, 11 August 2012 at 00:24:40 UTC, dsimcha wrote:



 I was considering writing one, but I wanted it to be high-level=20
 and easy-to-use.  I ended up not doing it, initially because I=20
 was waiting for serialization to be added to Phobos (which I=20
 thought was imminent) and then because I got busy with=20
 unrelated things.




Easy to use is good. Did you survey Boost.MPI?


 I think that a nice high-level MPI wrapper for D should be=20
 tightly integrated into a serialization library to encapsulate=20
 the low-level details of passing non-trivial data structures=20
 across nodes.  I doubt I'll get around to implementing it when=20
 serialization is added, though, because I'm probably past the=20
 MPI-using stage of my life (my Ph.D. research is basically=20
 finished, I'm just revising my dissertation and preparing to=20
 defend) so I wouldn't get to eat my own dogfood.




Marshalling and unmarshalling data structures is a real pain. On the
other hand isn't it something that has to be explicit when working with
MPI since the MPI API works solely in machine level data types.


 Well, my PhD research is just beginning... :)
=20
 Any chance you could pass on what you have?  It might help me out
 a bit, and reduce my workload toward creating a usable MPI
 library.


I don't have any D-related material, and I don't use MPI for big
cluster-based codes, but I use MPI a lot for some small algorithm
benchmarks. So if you need someone to bounce idea off, feel free to
include me in the set of people you drop a line to.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Aug 11 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-08-11 12:30, Russel Winder wrote:


 Marshalling and unmarshalling data structures is a real pain. On the
 other hand isn't it something that has to be explicit when working with
 MPI since the MPI API works solely in machine level data types.


Probably not exactly what's needed but:

https://github.com/jacob-carlborg/orange

-- 
/Jacob Carlborg
Aug 11 2012
prev sibling parent Russel Winder <russel winder.org.uk> writes:
On Sat, 2012-08-11 at 02:24 +0200, dsimcha wrote:
[=E2=80=A6]

 serialization is added, though, because I'm probably past the=20
 MPI-using stage of my life (my Ph.D. research is basically=20
 finished, I'm just revising my dissertation and preparing to=20
 defend) so I wouldn't get to eat my own dogfood.


Good luck with finishing the dissertation and the defence.

--=20
Russel.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder ekiga.n=
et
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel winder.org.uk
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder
Aug 11 2012