digitalmars.D.learn - core.sync.rwmutex example
- Charles Hixson via Digitalmars-d-learn (29/29) May 09 2014 The example code from core.sync.rwmutex seems bugged. After copying it
- Joshua Niehus (44/44) May 09 2014 Hi Charles,
- Charles Hixson via Digitalmars-d-learn (63/107) May 09 2014 Thank you very much for the response.
- TheFlyingFiddle (18/27) May 10 2014 If you are worried that other messages having the same signature
- Charles Hixson via Digitalmars-d-learn (36/63) May 12 2014 I think you're probably right. It should be a fairly fast lookup
The example code from core.sync.rwmutex seems bugged. After copying it
I added an import for core.sync.rwmutex, and moved the executions of
runTest into...well:
void main()
{ runTest(ReadWriteMutex.Policy.PREFER_READERS);
runTest(ReadWriteMutex.Policy.PREFER_WRITERS);
}
Then I tried to compile it. I got the following error messages:
test3.d(36): Error: class core.sync.rwmutex.ReadWriteMutex member
m_commonMutex is not accessible
test3.d(38): Error: class core.sync.rwmutex.ReadWriteMutex member
m_numQueuedReaders is not accessible
test3.d(39): Error: class core.sync.rwmutex.ReadWriteMutex member
m_numQueuedWriters is not accessible
Checking out the documentation, I don't see that they SHOULD be
accessible, so I think the compiler's correct, and the example is wrong.
P.S.: Does anyone have a good working example of rwmutex? I'm trying
to build a hash table that is write accessible from one thread, and
readable from anywhere, and this looked like the best way to do
it...except that when I start to figure out how to use it I get errors.
Also does anyone have any examples of two directional communication
between two particular threads (a bit more than just yes/no) in the
presence of multiple other threads, so that when a thread asks another
for information, only that other thread is allowed to reply? Perhaps
that's a better way to implement the shared-read hash table. (I'd like
to use std.concurrency, but I can't figure out how one is supposed to
manage specific inter-thread communications.)
--
Charles Hixson
May 09 2014
Hi Charles,
would the following work (just a shot in the dark) ?
//---
module test;
import std.stdio;
import std.concurrency;
void spawnedFuncFoo(Tid tid, Tid tidBar) {
receive(
(int i) {
writeln("Foo Received the number ", i);
send(tidBar, i, thisTid);
auto barSuccessful = receiveOnly!(string);
writeln("Bar got my (Foo) message");
}
);
send(tid, true);
}
void spawnedFuncBar(Tid tid) {
receive(
(int i, Tid tidFoo) {
writeln("Foo passed me (Bar) the number ", i);
send(tidFoo, "done");
}
);
receive(
(string sig) {
writeln("Main says I'm (Bar) done.");
send(tid, 42);
}
);
}
void main() {
auto tidBar = spawn(&spawnedFuncBar, thisTid);
auto tidFoo = spawn(&spawnedFuncFoo, thisTid, tidBar);
send(tidFoo, 42);
auto fooWasSuccessful = receiveOnly!(bool);
assert(fooWasSuccessful);
send(tidBar, "your done");
auto barWasSuccessful = receiveOnly!(int);
assert(barWasSuccessful == 42);
writeln("Successfully had two separate threads communicate
with each other");
}
//---
May 09 2014
On 05/09/2014 02:51 PM, Joshua Niehus via Digitalmars-d-learn wrote:
Hi Charles,
would the following work (just a shot in the dark) ?
//---
module test;
import std.stdio;
import std.concurrency;
void spawnedFuncFoo(Tid tid, Tid tidBar) {
receive(
(int i) {
writeln("Foo Received the number ", i);
send(tidBar, i, thisTid);
auto barSuccessful = receiveOnly!(string);
writeln("Bar got my (Foo) message");
}
);
send(tid, true);
}
void spawnedFuncBar(Tid tid) {
receive(
(int i, Tid tidFoo) {
writeln("Foo passed me (Bar) the number ", i);
send(tidFoo, "done");
}
);
receive(
(string sig) {
writeln("Main says I'm (Bar) done.");
send(tid, 42);
}
);
}
void main() {
auto tidBar = spawn(&spawnedFuncBar, thisTid);
auto tidFoo = spawn(&spawnedFuncFoo, thisTid, tidBar);
send(tidFoo, 42);
auto fooWasSuccessful = receiveOnly!(bool);
assert(fooWasSuccessful);
send(tidBar, "your done");
auto barWasSuccessful = receiveOnly!(int);
assert(barWasSuccessful == 42);
writeln("Successfully had two separate threads communicate
with each other");
}
//---
Thank you very much for the response.
How do you ensure that the received message (in "auto barWasSuccessful =
receiveOnly!(int); ") came from the thread sent to by tidFoo? Also, why
two receives in
spawnedFuncBar? I thought that receive handled multiple different
arguments with a single receive? Are you asserting that the arguments
happen in sequence, as it appears? That doesn't look as if it would
scale will to multiple threads. And with multiple threads I would
expect the mailbox to start getting very full with inappropriate
messages that never got handled, because they came at the wrong time
(and arguably from the wrong sender, as it seems that there is an
assumption that there is no intervening thread sending any messages).
That's why I was looking for a way to ensure that the message being
received came from the appropriate sender. You can't depend on messages
to always come from the source you expect. This probably means that you
need to have only ONE receive statement, located in a loop. And some
way to check and set the status of incoming messages. Also a yield
method that lets one receive and stack any incoming messages during a
busy cycle, though I think that that's what receive does.
OTOH, I do note that you are using "auto barWasSuccessful =
receiveOnly!(int);" which implies that the value returned is not a void,
even though you never use the result. This fits with the claims of
TDPL, but make me increasingly dubious about the documentation. Or do I
just not understand what a function of type void is supposed to return?
The hashtable itself will be a uint64_t[string] which is confined to one
thread, so the access methods need to be rather straightforwards. I'm
thinking of:
receive
( (string s, uint64_t id) { void set(s, id); }
(string s) { void get(s); }
(Variant v) { void reportError (v); }
);
get should send the id back to the calling thread. set updates the AA
and sends true or false (success) back to the calling thread. But
doesn't this design lock the entire hashtable while the update is in
progress? Is there a better way? set will be, approx:
void set (string s, uint64_t id)
{ tbl[s] = id;
send (tid, true);
}
etc. But I'm worried about the receiving end. It needs, somehow, to
ensure that the message it receives is the appropriate message, and that
other messages don't get dropped while it's waiting for the answer...or,
perhaps worse, substituted for the expected answer. If I can depend on
msg[0] of "auto msg = receiveOnly!(Tid, bool)" that will allow me to
check that the message was received from the proper source Also, if a
message is received that if of an inappropriate form, an exception is
thrown. Recovery looks problematic. But this is a case that should be
expected when different threads are sending messages. And if the
message is of the correct form, but from the wrong thread, how can one
re-queue it for later processing?
So far the best idea I've got is to have a dequeue of messages that are
stacked with "receiveOnly!Variant". But this would need to be accesses
in various places throughout the thread, and the handling looks messy.
OTOH, it does let me save messages that come in at the wrong time where
I can deal with them later. So it's not all bad. But it gives up on the
processing that receive makes available. And note that this processing
doesn't need to be done in the hashtable function, that's pretty clear,
but wherever it's used. Which is terrible. As it is, I might be better
putting the hashtable in s synchronized class. The entire hashtable is
getting locked on every read anyway.
-- Charles Hixson
May 09 2014
On Friday, 9 May 2014 at 23:12:44 UTC, Charles Hixson via Digitalmars-d-learn wrote:But I'm worried about the receiving end. It needs, somehow, to ensure that the message it receives is the appropriate message, and that other messages don't get dropped while it's waiting for the answer...or, perhaps worse, substituted for the expected answer. If I can depend on msg[0] of "auto msg = receiveOnly!(Tid, bool)" that will allow me to check that the message was received from the proper sourceIf you are worried that other messages having the same signature will be sent from other sources than the expected source you could make use of message tagging. Simply wrap the boolean result in a struct with a descriptive name. struct SharedHashMapSetCB { bool flag; } void set (string s, uint64_t id) { tbl[s] = id; send (SharedHashMapSetCB(true)); } //On the receiving end auto msg = receiveOnly!SharedHashMapSetCB();But doesn't this design lock the entire hash-table while the update is in progress? Is there a better way?I think a shared memory hash-map is better for your use case. Working with message passing is preferable done asynchronously. Blocking calls (send followed by receive) is likely to be slower then simply waiting on a semaphore.
May 10 2014
Charles Hixson via Digitalmars-d-learn On 05/10/2014 06:28 PM, TheFlyingFiddle via Digitalmars-d-learn wrote:On Friday, 9 May 2014 at 23:12:44 UTC, Charles Hixson via Digitalmars-d-learn wrote:I think you're probably right. It should be a fairly fast lookup anyway, and since there should be only one copy of the table, I don't think one would get much of a speedup no matter what one did. OTOH, it can't be directly shared. What can be shared is a handle to a singleton synchronized class. Its requirement that it can be updated, IIUC, means that it can't be directly shared. OTOH... I think I may need to build a mailbox class, the design I'm not yet sure about, that acts as an interface between the major number of threads and each other. Access to it needs to be non-blocking (essentially) so it had probably better ONLY act as a message switchboard. I want it to pass messages with "time" and "origin" (thread id) prefixed to them. OTOH, I probably only need to support a small number of message formats, perhaps two, perhaps three. And it would be more efficient if whenever a thread polled it, it responded with all waiting mail for that destination. So it's only sending in response to a received message directed at it, but it holds all messages directed at some other thread. This allows the other threads to loop through their processing, only occasionally pausing to check whether they have any mail or not. I'm hoping a couple of ms will be a long enough timeout. The mailbox could spend most of it's time sitting blocked at receive. This isn't the design I was thinking about earlier, but it seems better, and it's the design I originally ended up with the last time I thought about this problem seriously, though admittedly I never got around to implementing it. The question in my mind is "is it better to build this on top of std.concurrency, or to go back to the thread class. The code of std.concurrency shows pretty clearly how to build a mailbox, but the one that it builds doesn't store the things that I want, and is a lot more flexible than I have any use for. Still, it's being maintained as a part of the language...but it's a private class, so I can't use it directly. But threading is difficult and error prone, so it might be better to write a layer on top of std.concurrency, even if it means I need to re-implement a lot of what has already been done. -- Charles HixsonBut I'm worried about the receiving end. It needs, somehow, to ensure that the message it receives is the appropriate message, and that other messages don't get dropped while it's waiting for the answer...or, perhaps worse, substituted for the expected answer. If I can depend on msg[0] of "auto msg = receiveOnly!(Tid, bool)" that will allow me to check that the message was received from the proper sourceIf you are worried that other messages having the same signature will be sent from other sources than the expected source you could make use of message tagging. Simply wrap the boolean result in a struct with a descriptive name. struct SharedHashMapSetCB { bool flag; } void set (string s, uint64_t id) { tbl[s] = id; send (SharedHashMapSetCB(true)); } //On the receiving end auto msg = receiveOnly!SharedHashMapSetCB();But doesn't this design lock the entire hash-table while the update is in progress? Is there a better way?I think a shared memory hash-map is better for your use case. Working with message passing is preferable done asynchronously. Blocking calls (send followed by receive) is likely to be slower then simply waiting on a semaphore.
May 12 2014