www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Some kind of RPC exists for D?

reply "Bienlein" <jeti789 web.de> writes:
Hello,

I'm looking for a way to do some kind of RPC in D. Some way of 
being able to say aFoo.bar(int i, ...) with receiver object and 
method being marshalled at the sender's site and being 
unmarshalled and invoked at the receiver's site. Any hints 
appreciated.

Thanks, Bienlein
Jun 18 2014
next sibling parent "Dicebot" <public dicebot.lv> writes:
On Wednesday, 18 June 2014 at 19:24:03 UTC, Bienlein wrote:

 Hello,

 I'm looking for a way to do some kind of RPC in D. Some way of 
 being able to say aFoo.bar(int i, ...) with receiver object and 
 method being marshalled at the sender's site and being 
 unmarshalled and invoked at the receiver's site. Any hints 
 appreciated.

 Thanks, Bienlein


vibe.web.rest server/client combo is effectively RPC over 
HTTP/json
Jun 18 2014
prev sibling next sibling parent "David Nadlinger" <code klickverbot.at> writes:
On Wednesday, 18 June 2014 at 19:24:03 UTC, Bienlein wrote:

 I'm looking for a way to do some kind of RPC in D. Some way of 
 being able to say aFoo.bar(int i, ...) with receiver object and 
 method being marshalled at the sender's site and being 
 unmarshalled and invoked at the receiver's site. Any hints 
 appreciated.


I wrote an implementation of Thrift for D a while back.

It supports RPC and marshalling of arbitrary objects within the 
boundaries set by the Thrift protocol (no classes, no pointers, 
…).

David
Jun 18 2014
prev sibling next sibling parent Rikki Cattermole <alphaglosined gmail.com> writes:
On 19/06/2014 7:24 a.m., Bienlein wrote:

 Hello,

 I'm looking for a way to do some kind of RPC in D. Some way of being
 able to say aFoo.bar(int i, ...) with receiver object and method being
 marshalled at the sender's site and being unmarshalled and invoked at
 the receiver's site. Any hints appreciated.

 Thanks, Bienlein


Just for reference sake, I'm also working on something similar via an 
actor framework [0].

[0] https://github.com/rikkimax/dakka/wiki/Protocol
Jun 18 2014
prev sibling next sibling parent reply "Bienlein" <jeti789 web.de> writes:
What I want to do is some little framework that allows 
Julia-style (julialang.org) distributed computation: send some 
function invocation to some other machine along with the numbers 
to be crunched and get the result back.


 vibe.web.rest server/client combo is effectively RPC over 
 HTTP/json


This looks good. For what am I'm thinking of doing performance is 
important. In that way rest makes me think a bit or is this only 
a prejudice from the Java world?


 I wrote an implementation of Thrift for D a while back


This also looks interesting. Because my C/C++/D skills are 
limited being a Smalltalk/Java developer (only played with C++ 
when studying) I have to stick to what is easier to use. It would 
be a fun & leisure & learning project anyway...

Regards, Bienlein
Jun 19 2014
parent reply "Dicebot" <public dicebot.lv> writes:
On Thursday, 19 June 2014 at 10:25:08 UTC, Bienlein wrote:

 vibe.web.rest server/client combo is effectively RPC over 
 HTTP/json


 This looks good. For what am I'm thinking of doing performance 
 is important. In that way rest makes me think a bit or is this 
 only a prejudice from the Java world?


It is not very suitable for applications where response latency 
is important because HTTP itself and JSON (de)serialization 
create considerable needless overhead (compared to something like 
thrift).

However if you application design implies thousands of RPC calls 
per second it is quite likely performance won't be good with any 
RPC implementation :)
Jun 19 2014
parent reply "David Nadlinger" <code klickverbot.at> writes:
On Thursday, 19 June 2014 at 12:13:12 UTC, Dicebot wrote:

 However if you application design implies thousands of RPC 
 calls per second it is quite likely performance won't be good 
 with any RPC implementation :)


Backend people at Google, Facebook, and others would beg to 
disagree. ;)

Of course, the calls counted here are from multiple clients to a 
single server. If your fan-out is in the thousands, then you 
indeed have a problem.

David
Jun 19 2014
parent "Dicebot" <public dicebot.lv> writes:
On Thursday, 19 June 2014 at 13:54:01 UTC, David Nadlinger wrote:

 On Thursday, 19 June 2014 at 12:13:12 UTC, Dicebot wrote:

 However if you application design implies thousands of RPC 
 calls per second it is quite likely performance won't be good 
 with any RPC implementation :)


 Backend people at Google, Facebook, and others would beg to 
 disagree. ;)

 Of course, the calls counted here are from multiple clients to 
 a single server. If your fan-out is in the thousands, then you 
 indeed have a problem.

 David


Yes, you are correct, I have indeed meant bi-direction 
distributed communication model, something similar to what Erlang 
provides out of the box. Of course in classical client-server 
model handling even tens of thousands of RPC calls per second is 
feasible.
Jun 19 2014
prev sibling next sibling parent reply "Kagamin" <spam here.lot> writes:
On Wednesday, 18 June 2014 at 19:24:03 UTC, Bienlein wrote:

 Hello,

 I'm looking for a way to do some kind of RPC in D. Some way of 
 being able to say aFoo.bar(int i, ...) with receiver object and 
 method being marshalled at the sender's site and being 
 unmarshalled and invoked at the receiver's site. Any hints 
 appreciated.

 Thanks, Bienlein


What data load profile do you expect? Vibe is tuned to handle 
thousands simultaneous incoming light requests (milliseconds), 
while distributed computing works better with exclusive heavy 
requests, at least minutes of work worth, BOINC uses hours worth 
work items.
Jun 19 2014
parent reply "Bienlein" <jeti789 web.de> writes:
 What data load profile do you expect? Vibe is tuned to handle 
 thousands simultaneous incoming light requests (milliseconds), 
 while distributed computing works better with exclusive heavy 
 requests, at least minutes of work worth, BOINC uses hours 
 worth work items.


Communication will be bi-directional as some thread sends 
computation along with the data to be computed to some remote 
thread and somewhen waits for the answer. The amount of data will 
be relatively small, that is only consisting of numbers to be 
processed creating some result, because Julia is geared towards 
scientific computing. In that way there is little data compared 
to enterprise computing.

How often remote computations are initiated depends to a large 
extend on the user. So that is hard to say. But from the typical 
use case scenario it will be anything but few large messages to 
go across the wire. I might have to do a bit more research 
looking at Julia to get a better understanding. But it's all 
about performance, e.g. crunching as many numbers as quickly as 
possible. So I will have to stick to a high-performance solution 
anyway. For that reason it looks more like Swift or ZeroMQ. 
Depends a lot which one can do marshalling and unmarshalling of 
the function invocation and the data more transparently than the 
other.
Jun 20 2014
parent "Dicebot" <public dicebot.lv> writes:
On Friday, 20 June 2014 at 08:05:01 UTC, Bienlein wrote:

 What data load profile do you expect? Vibe is tuned to handle 
 thousands simultaneous incoming light requests (milliseconds), 
 while distributed computing works better with exclusive heavy 
 requests, at least minutes of work worth, BOINC uses hours 
 worth work items.


 Communication will be bi-directional as some thread sends 
 computation along with the data to be computed to some remote 
 thread and somewhen waits for the answer. The amount of data 
 will be relatively small, that is only consisting of numbers to 
 be processed creating some result, because Julia is geared 
 towards scientific computing. In that way there is little data 
 compared to enterprise computing.

 How often remote computations are initiated depends to a large 
 extend on the user. So that is hard to say. But from the 
 typical use case scenario it will be anything but few large 
 messages to go across the wire. I might have to do a bit more 
 research looking at Julia to get a better understanding. But 
 it's all about performance, e.g. crunching as many numbers as 
 quickly as possible. So I will have to stick to a 
 high-performance solution anyway. For that reason it looks more 
 like Swift or ZeroMQ. Depends a lot which one can do 
 marshalling and unmarshalling of the function invocation and 
 the data more transparently than the other.


application performance != networking performance. Sounds like in 
your use case actual communication overhead has low impact 
compared to actual computations done remotely. In that case 
vibe.d solution can actually be very helpful because it will 
provide async communication model out of the box (your nodes will 
be able to do some other useful work while waiting for responses 
from other nodes).
Jun 20 2014
prev sibling parent reply "Paul" <aquagnu gmail.com> writes:
On Wednesday, 18 June 2014 at 19:24:03 UTC, Bienlein wrote:

 Hello,

 I'm looking for a way to do some kind of RPC in D. Some way of 
 being able to say aFoo.bar(int i, ...) with receiver object and 
 method being marshalled at the sender's site and being 
 unmarshalled and invoked at the receiver's site. Any hints 
 appreciated.

 Thanks, Bienlein


I wrote some (JSONRPC and TXTRPC) and used them with vibe. I can 
send you via email
Jun 21 2014
parent "Bienlein" <jeti789 web.de> writes:
On Saturday, 21 June 2014 at 18:22:54 UTC, Paul wrote:


 I wrote some (JSONRPC and TXTRPC) and used them with vibe. I 
 can send you via email


Hi Paul, alternatively you could upload your code to GitHub or 
some similar place and place the link here. Then you also have a 
means to "proof" that the code is your work along with the date 
of publication.

-- Bienlein
Jun 22 2014