www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - core.sync.rwmutex example

reply Charles Hixson via Digitalmars-d-learn writes:
The example code from core.sync.rwmutex seems bugged.  After copying it 
I added an import for core.sync.rwmutex, and moved the executions of 
runTest into...well:
void main()
{    runTest(ReadWriteMutex.Policy.PREFER_READERS);
     runTest(ReadWriteMutex.Policy.PREFER_WRITERS);
}

Then I tried to compile it.  I got the following error messages:
test3.d(36): Error: class core.sync.rwmutex.ReadWriteMutex member 
m_commonMutex is not accessible
test3.d(38): Error: class core.sync.rwmutex.ReadWriteMutex member 
m_numQueuedReaders is not accessible
test3.d(39): Error: class core.sync.rwmutex.ReadWriteMutex member 
m_numQueuedWriters is not accessible

Checking out the documentation, I don't see that they SHOULD be 
accessible, so I think the compiler's correct, and the example is wrong.

P.S.:  Does anyone have a good working example of rwmutex?  I'm trying 
to build a hash table that is write accessible from one thread, and 
readable from anywhere, and this looked like the best way to do 
it...except that when I start to figure out how to use it I get errors.

Also does anyone have any examples of two directional communication 
between two particular threads (a bit more than just yes/no) in the 
presence of multiple other threads, so that when a thread asks another 
for information, only that other thread  is allowed to reply?  Perhaps 
that's a better way to implement the shared-read hash table.  (I'd like 
to use std.concurrency, but I can't figure out how one is supposed to 
manage specific inter-thread communications.)

-- 
Charles Hixson
May 09 2014
parent reply "Joshua Niehus" <jm.niehus gmail.com> writes:
Hi Charles,

would the following work (just a shot in the dark) ?

//---
module test;

import std.stdio;
import std.concurrency;

void spawnedFuncFoo(Tid tid, Tid tidBar) {
      receive(
          (int i) {
              writeln("Foo Received the number ", i);
              send(tidBar, i, thisTid);
              auto barSuccessful = receiveOnly!(string);
              writeln("Bar got my (Foo) message");
          }
      );

      send(tid, true);
}

void spawnedFuncBar(Tid tid) {
      receive(
          (int i, Tid tidFoo) {
              writeln("Foo passed me (Bar) the number ", i);
              send(tidFoo, "done");
          }
      );

      receive(
          (string sig) {
              writeln("Main says I'm (Bar) done.");
              send(tid, 42);
          }
      );
}

void main() {
      auto tidBar = spawn(&spawnedFuncBar, thisTid);
      auto tidFoo = spawn(&spawnedFuncFoo, thisTid, tidBar);
      send(tidFoo, 42);
      auto fooWasSuccessful = receiveOnly!(bool);
      assert(fooWasSuccessful);

      send(tidBar, "your done");
      auto barWasSuccessful = receiveOnly!(int);
      assert(barWasSuccessful == 42);
      writeln("Successfully had two separate threads communicate
with each other");
}
//---
May 09 2014
parent reply Charles Hixson via Digitalmars-d-learn writes:
On 05/09/2014 02:51 PM, Joshua Niehus via Digitalmars-d-learn wrote:
 Hi Charles,

 would the following work (just a shot in the dark) ?

 //---
 module test;

 import std.stdio;
 import std.concurrency;

 void spawnedFuncFoo(Tid tid, Tid tidBar) {
      receive(
          (int i) {
              writeln("Foo Received the number ", i);
              send(tidBar, i, thisTid);
              auto barSuccessful = receiveOnly!(string);
              writeln("Bar got my (Foo) message");
          }
      );

      send(tid, true);
 }

 void spawnedFuncBar(Tid tid) {
      receive(
          (int i, Tid tidFoo) {
              writeln("Foo passed me (Bar) the number ", i);
              send(tidFoo, "done");
          }
      );

      receive(
          (string sig) {
              writeln("Main says I'm (Bar) done.");
              send(tid, 42);
          }
      );
 }

 void main() {
      auto tidBar = spawn(&spawnedFuncBar, thisTid);
      auto tidFoo = spawn(&spawnedFuncFoo, thisTid, tidBar);
      send(tidFoo, 42);
      auto fooWasSuccessful = receiveOnly!(bool);
      assert(fooWasSuccessful);

      send(tidBar, "your done");
      auto barWasSuccessful = receiveOnly!(int);
      assert(barWasSuccessful == 42);
      writeln("Successfully had two separate threads communicate
 with each other");
 }
 //---
Thank you very much for the response. How do you ensure that the received message (in "auto barWasSuccessful = receiveOnly!(int); ") came from the thread sent to by tidFoo? Also, why two receives in spawnedFuncBar? I thought that receive handled multiple different arguments with a single receive? Are you asserting that the arguments happen in sequence, as it appears? That doesn't look as if it would scale will to multiple threads. And with multiple threads I would expect the mailbox to start getting very full with inappropriate messages that never got handled, because they came at the wrong time (and arguably from the wrong sender, as it seems that there is an assumption that there is no intervening thread sending any messages). That's why I was looking for a way to ensure that the message being received came from the appropriate sender. You can't depend on messages to always come from the source you expect. This probably means that you need to have only ONE receive statement, located in a loop. And some way to check and set the status of incoming messages. Also a yield method that lets one receive and stack any incoming messages during a busy cycle, though I think that that's what receive does. OTOH, I do note that you are using "auto barWasSuccessful = receiveOnly!(int);" which implies that the value returned is not a void, even though you never use the result. This fits with the claims of TDPL, but make me increasingly dubious about the documentation. Or do I just not understand what a function of type void is supposed to return? The hashtable itself will be a uint64_t[string] which is confined to one thread, so the access methods need to be rather straightforwards. I'm thinking of: receive ( (string s, uint64_t id) { void set(s, id); } (string s) { void get(s); } (Variant v) { void reportError (v); } ); get should send the id back to the calling thread. set updates the AA and sends true or false (success) back to the calling thread. But doesn't this design lock the entire hashtable while the update is in progress? Is there a better way? set will be, approx: void set (string s, uint64_t id) { tbl[s] = id; send (tid, true); } etc. But I'm worried about the receiving end. It needs, somehow, to ensure that the message it receives is the appropriate message, and that other messages don't get dropped while it's waiting for the answer...or, perhaps worse, substituted for the expected answer. If I can depend on msg[0] of "auto msg = receiveOnly!(Tid, bool)" that will allow me to check that the message was received from the proper source Also, if a message is received that if of an inappropriate form, an exception is thrown. Recovery looks problematic. But this is a case that should be expected when different threads are sending messages. And if the message is of the correct form, but from the wrong thread, how can one re-queue it for later processing? So far the best idea I've got is to have a dequeue of messages that are stacked with "receiveOnly!Variant". But this would need to be accesses in various places throughout the thread, and the handling looks messy. OTOH, it does let me save messages that come in at the wrong time where I can deal with them later. So it's not all bad. But it gives up on the processing that receive makes available. And note that this processing doesn't need to be done in the hashtable function, that's pretty clear, but wherever it's used. Which is terrible. As it is, I might be better putting the hashtable in s synchronized class. The entire hashtable is getting locked on every read anyway. -- Charles Hixson
May 09 2014
parent reply "TheFlyingFiddle" <kurtyan student.chalmers.se> writes:
On Friday, 9 May 2014 at 23:12:44 UTC, Charles Hixson via 
Digitalmars-d-learn wrote:

 But I'm worried about the receiving end.  It needs, somehow, to 
 ensure that the message it receives is the appropriate message, 
 and that other messages don't get dropped while it's waiting 
 for the answer...or, perhaps worse, substituted for the 
 expected answer.  If I can depend on msg[0] of "auto msg = 
 receiveOnly!(Tid, bool)" that will allow me to check that the 
 message was received from the proper source
If you are worried that other messages having the same signature will be sent from other sources than the expected source you could make use of message tagging. Simply wrap the boolean result in a struct with a descriptive name. struct SharedHashMapSetCB { bool flag; } void set (string s, uint64_t id) { tbl[s] = id; send (SharedHashMapSetCB(true)); } //On the receiving end auto msg = receiveOnly!SharedHashMapSetCB();
 But doesn't this design lock the entire hash-table while the 
 update is in progress?  Is there a better way?
I think a shared memory hash-map is better for your use case. Working with message passing is preferable done asynchronously. Blocking calls (send followed by receive) is likely to be slower then simply waiting on a semaphore.
May 10 2014
parent Charles Hixson via Digitalmars-d-learn writes:
On 05/10/2014 06:28 PM, TheFlyingFiddle via Digitalmars-d-learn wrote:
 On Friday, 9 May 2014 at 23:12:44 UTC, Charles Hixson via 
 Digitalmars-d-learn wrote:

 But I'm worried about the receiving end. It needs, somehow, to ensure 
 that the message it receives is the appropriate message, and that 
 other messages don't get dropped while it's waiting for the 
 answer...or, perhaps worse, substituted for the expected answer.  If 
 I can depend on msg[0] of "auto msg = receiveOnly!(Tid, bool)" that 
 will allow me to check that the message was received from the proper 
 source
If you are worried that other messages having the same signature will be sent from other sources than the expected source you could make use of message tagging. Simply wrap the boolean result in a struct with a descriptive name. struct SharedHashMapSetCB { bool flag; } void set (string s, uint64_t id) { tbl[s] = id; send (SharedHashMapSetCB(true)); } //On the receiving end auto msg = receiveOnly!SharedHashMapSetCB();
 But doesn't this design lock the entire hash-table while the update 
 is in progress?  Is there a better way?
I think a shared memory hash-map is better for your use case. Working with message passing is preferable done asynchronously. Blocking calls (send followed by receive) is likely to be slower then simply waiting on a semaphore.
I think you're probably right. It should be a fairly fast lookup anyway, and since there should be only one copy of the table, I don't think one would get much of a speedup no matter what one did. OTOH, it can't be directly shared. What can be shared is a handle to a singleton synchronized class. Its requirement that it can be updated, IIUC, means that it can't be directly shared. OTOH... I think I may need to build a mailbox class, the design I'm not yet sure about, that acts as an interface between the major number of threads and each other. Access to it needs to be non-blocking (essentially) so it had probably better ONLY act as a message switchboard. I want it to pass messages with "time" and "origin" (thread id) prefixed to them. OTOH, I probably only need to support a small number of message formats, perhaps two, perhaps three. And it would be more efficient if whenever a thread polled it, it responded with all waiting mail for that destination. So it's only sending in response to a received message directed at it, but it holds all messages directed at some other thread. This allows the other threads to loop through their processing, only occasionally pausing to check whether they have any mail or not. I'm hoping a couple of ms will be a long enough timeout. The mailbox could spend most of it's time sitting blocked at receive. This isn't the design I was thinking about earlier, but it seems better, and it's the design I originally ended up with the last time I thought about this problem seriously, though admittedly I never got around to implementing it. The question in my mind is "is it better to build this on top of std.concurrency, or to go back to the thread class. The code of std.concurrency shows pretty clearly how to build a mailbox, but the one that it builds doesn't store the things that I want, and is a lot more flexible than I have any use for. Still, it's being maintained as a part of the language...but it's a private class, so I can't use it directly. But threading is difficult and error prone, so it might be better to write a layer on top of std.concurrency, even if it means I need to re-implement a lot of what has already been done. -- Charles Hixson
May 12 2014