www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Passing associative array to another thread

reply Martin Drasar <drasar ics.muni.cz> writes:
Hi,

I am using the std.concurrency module and I would like to send an
associative array to another thread.

If I try this:

string[string] aa;
someThread.send(aa);

I get: Aliases to mutable thread-local data not allowed.

And if I try to use this:

immutable(string[string]) aa;
someThread.send(aa);

I get:
/usr/include/d/dmd/phobos/std/variant.d(539): Error: *p is not mutable

which is because the send() creates a Message struct that stores the
data in a Variant.

And now I am stuck, because I do not have any idea what to do. Any advice?

Thanks,
Martin
Sep 21 2012
next sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-09-21 16:33, Martin Drasar wrote:

 Hi,

 I am using the std.concurrency module and I would like to send an
 associative array to another thread.

 If I try this:

 string[string] aa;
 someThread.send(aa);

 I get: Aliases to mutable thread-local data not allowed.

 And if I try to use this:

 immutable(string[string]) aa;
 someThread.send(aa);

 I get:
 /usr/include/d/dmd/phobos/std/variant.d(539): Error: *p is not mutable

 which is because the send() creates a Message struct that stores the
 data in a Variant.

 And now I am stuck, because I do not have any idea what to do. Any advice?


Perhaps declaring the associative array as "shared". An alternative 
would be to serialize the aa, pass it to another thread, and deserialize 
it. That would though create a copy.

-- 
/Jacob Carlborg
Sep 21 2012
next sibling parent reply Martin Drasar <drasar ics.muni.cz> writes:
On 21.9.2012 19:01, Jacob Carlborg wrote:

 Perhaps declaring the associative array as "shared". An alternative
 would be to serialize the aa, pass it to another thread, and deserialize
 it. That would though create a copy.


Hi Jacob,

thanks for the hint. Making it shared sounds a bit fishy to me. My
intention is to pass some read only data, that are in fact thread local
and there is no real need to make them shared.

The (de)serialization is possible but the overhead seems a bit
pointless. I will alter the code to use something else than AAs if there
is no other way.

The data I am trying to pass is in fact just name-value pairs. I have
tried to use Tuples, but I have hit another batch of problems. One was


This compiles ok:
struct S { Tuple!int i; }

This does not:
struct S {
  Tuple!int i;
  SysTime   t;
}

Error: function
std.typecons.Tuple!(int).Tuple.opEquals!(const(Tuple!(int))).opEquals
(const(Tuple!(int)) rhs) is not callable using argument types
(const(Tuple!(int))) const


shows up with the SysTime in place...

Martin
Sep 22 2012
parent reply Jacob Carlborg <doob me.com> writes:
On 2012-09-22 11:24, Martin Drasar wrote:


 thanks for the hint. Making it shared sounds a bit fishy to me. My
 intention is to pass some read only data, that are in fact thread local
 and there is no real need to make them shared.


The whole point of thread local data is that it's only accessible from a 
single thread. If you want to share it with another thread you have, as 
far as I know, there options:

1. Declare it as "shared"
2. Declare it as "immutable"
3. Make a copy, i.e. serialize the data


 The (de)serialization is possible but the overhead seems a bit
 pointless. I will alter the code to use something else than AAs if there
 is no other way.

 The data I am trying to pass is in fact just name-value pairs. I have
 tried to use Tuples, but I have hit another batch of problems. One was



Looking at your original example I don't understand why the immutable aa 
won't work. That's the whole point of immutable, it's safe to share 
among threads. It's probably a bug somewhere. I think someone else can 
answer these questions better than me.

-- 
/Jacob Carlborg
Sep 22 2012
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Saturday, September 22, 2012 12:30:30 Jacob Carlborg wrote:

 Looking at your original example I don't understand why the immutable aa
 won't work. That's the whole point of immutable, it's safe to share
 among threads. It's probably a bug somewhere. I think someone else can
 answer these questions better than me.


The problem with immutable is probably due to this bug:

http://d.puremagic.com/issues/show_bug.cgi?id=5538

And casting to shared probably won't work due to this bug:

http://d.puremagic.com/issues/show_bug.cgi?id=6585

std.variant needs quite a bit of work done to it, and it's causing problems 
with std.concurrency is this case. In the interim, I suspect that just about 
the only way to get an AA across threads is to just make it shared and not use 
std.concurrency at all, as undesirable as that may be. Your serialization 
suggestion would probably be the only other choice, though that would require 
something like Orange, as Phobos doesn't have such facilities.

- Jonathan M Davis
Sep 22 2012
prev sibling next sibling parent Martin Drasar <drasar ics.muni.cz> writes:
On 22.9.2012 13:19, Jonathan M Davis wrote:

 The problem with immutable is probably due to this bug:
 
 http://d.puremagic.com/issues/show_bug.cgi?id=5538
 
 And casting to shared probably won't work due to this bug:
 
 http://d.puremagic.com/issues/show_bug.cgi?id=6585
 
 std.variant needs quite a bit of work done to it, and it's causing problems 
 with std.concurrency is this case. In the interim, I suspect that just about 
 the only way to get an AA across threads is to just make it shared and not use 
 std.concurrency at all, as undesirable as that may be. Your serialization 
 suggestion would probably be the only other choice, though that would require 
 something like Orange, as Phobos doesn't have such facilities.
 
 - Jonathan M Davis


Hi Jonathan,

I will work around the AA. As I have said, it is used only to pass
name-value pairs. So no need to ditch the entire std.concurrency because
of that.

Martin
Sep 22 2012
prev sibling parent reply Johannes Pfau <nospam example.com> writes:
Am Sat, 22 Sep 2012 12:30:30 +0200
schrieb Jacob Carlborg <doob me.com>:


 On 2012-09-22 11:24, Martin Drasar wrote:
 

 thanks for the hint. Making it shared sounds a bit fishy to me. My
 intention is to pass some read only data, that are in fact thread
 local and there is no real need to make them shared.


 
 The whole point of thread local data is that it's only accessible
 from a single thread. If you want to share it with another thread you
 have, as far as I know, there options:
 
 1. Declare it as "shared"


There's also __gshared.


 2. Declare it as "immutable"
 3. Make a copy, i.e. serialize the data
Sep 22 2012
next sibling parent Martin Drasar <drasar ics.muni.cz> writes:
On 22.9.2012 13:50, Johannes Pfau wrote:

 1. Declare it as "shared"


 There's also __gshared.


Yup, that works.

Thanks
Sep 22 2012
prev sibling parent Jacob Carlborg <doob me.com> writes:
On 2012-09-22 13:50, Johannes Pfau wrote:


 1. Declare it as "shared"


 There's also __gshared.


Yeah, forgot about that one.

-- 
/Jacob Carlborg
Sep 23 2012
prev sibling parent reply Sean Kelly <sean invisibleduck.org> writes:
On Sep 22, 2012, at 2:24 AM, Martin Drasar <drasar ics.muni.cz> wrote:


 On 21.9.2012 19:01, Jacob Carlborg wrote:

 Perhaps declaring the associative array as "shared". An alternative
 would be to serialize the aa, pass it to another thread, and =




deserialize

 it. That would though create a copy.


=20
 Hi Jacob,
=20
 thanks for the hint. Making it shared sounds a bit fishy to me. My
 intention is to pass some read only data, that are in fact thread =


local

 and there is no real need to make them shared.


If you're passing via std.concurrency then you'll currently have to cast =
to shared.  I'd been considering allowing Unique!T to be sent as well, =
but haven't done so yet.=
Sep 24 2012
parent Jacob Carlborg <doob me.com> writes:
On 2012-09-25 00:47, Sean Kelly wrote:


 If you're passing via std.concurrency then you'll currently have to cast to
shared.  I'd been considering allowing Unique!T to be sent as well, but haven't
done so yet.


Hey, if it's immutable why use std.concurrency at all? Just import the 
module and use the variable willy nilly. I mean, isn't that the whole 
point of immutable anyway, you can share it freely among threads without 
any risks?

-- 
/Jacob Carlborg
Sep 24 2012
prev sibling parent reply Jacob Carlborg <doob me.com> writes:
On 2012-09-21 16:33, Martin Drasar wrote:

 Hi,

 I am using the std.concurrency module and I would like to send an
 associative array to another thread.

 If I try this:

 string[string] aa;
 someThread.send(aa);

 I get: Aliases to mutable thread-local data not allowed.

 And if I try to use this:

 immutable(string[string]) aa;
 someThread.send(aa);

 I get:
 /usr/include/d/dmd/phobos/std/variant.d(539): Error: *p is not mutable


BTW, why do you need to use std.currency at all if it's immutable, just 
share it as a global. The whole point of immutable is that it can be 
freely shared among threads without any risks.

-- 
/Jacob Carlborg
Sep 25 2012
parent =?windows-1252?Q?Martin_Dra=9Aar?= <drasar ics.muni.cz> writes:
Dne 25.9.2012 18:19, Jacob Carlborg napsal(a):

 BTW, why do you need to use std.currency at all if it's immutable, just
 share it as a global. The whole point of immutable is that it can be
 freely shared among threads without any risks.


It is not some single piece of data.

I have a queue of tasks. Each task is a struct that contains among other 
things a user-supplied delegate and an AA with parameters for that delegate.

These tasks are created somewhere, inserted into the queue and then 
executed in parallel. Once created they are not modified anywhere and 
should exist only in one instance that is passed to that delegate.

It's just that I haven't found a better way to pass parameters that are 
unknown before to a delegate.

Martin
Sep 25 2012