www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Assoc Array for Concurrency

reply Chris <wendlec tcd.ie> writes:
What's the best way to make an assoc array fit for 
multi-threading? If this is not possible what would be the best 
alternative?

Say, for example, `data` is used by a class that is globally 
accessible to all threads. E.g. like this:

string[string] data;  // defined somewhere

public string getData(string key)
{
   if (key in data)
     return data[key];
   else
     return "";
}
Feb 29 2016
parent reply ZombineDev <petar.p.kirov gmail.com> writes:
On Monday, 29 February 2016 at 12:43:39 UTC, Chris wrote:

 What's the best way to make an assoc array fit for 
 multi-threading? If this is not possible what would be the best 
 alternative?

 Say, for example, `data` is used by a class that is globally 
 accessible to all threads. E.g. like this:

 string[string] data;  // defined somewhere

 public string getData(string key)
 {
   if (key in data)
     return data[key];
   else
     return "";
 }


I'm almost sure that built-in AAs don't provide automatic 
synchronization (in my tests I hit a deadlock), but you can 
easily wrap the AA into a struct that does the necessary 
synchronization. However be aware that even then you are not 
going to be safe, because more than one thread can try to access 
the keys or the values of the AA. Also note that because the 
built-in AAs rely on the GC, you may get poor scaling because 
every memory allocation can potentially take the global GC lock, 
which will block all threads from doing any work. So do your own 
tests and if you find the need to improve the performance, I 
would suggest investigating replacing the built-in AA with (for 
example) the hashmap from 
https://github.com/economicmodeling/containers in combination 
with a thread-local allocator.

Here's an example of how to wrap an AA into a moderately safe 
accessor. In the following example I create an additional thread 
which concurrently adds odd numbers into the AA, while the main 
threads add even nubmers:

http://dpaste.dzfl.pl/06025e6374eb

It segfaults on DPaste because you can't create threads there, 
however it works ok on my machine. The output look like this:
"0" : 0.140450140112896
"1" : 1.140450129700608
"2" : 2.140450140112896
"3" : 3.140450129700608
"4" : 4.140450140112896
"5" : 5.140450129700608
"6" : 6.140450140112896
"7" : 7.140450129700608
...
Feb 29 2016
parent Chris <wendlec tcd.ie> writes:
On Monday, 29 February 2016 at 17:38:11 UTC, ZombineDev wrote:

 On Monday, 29 February 2016 at 12:43:39 UTC, Chris wrote:

 [...]


 I'm almost sure that built-in AAs don't provide automatic 
 synchronization (in my tests I hit a deadlock), but you can 
 easily wrap the AA into a struct that does the necessary 
 synchronization. However be aware that even then you are not 
 going to be safe, because more than one thread can try to 
 access the keys or the values of the AA. Also note that because 
 the built-in AAs rely on the GC, you may get poor scaling 
 because every memory allocation can potentially take the global 
 GC lock, which will block all threads from doing any work. So 
 do your own tests and if you find the need to improve the 
 performance, I would suggest investigating replacing the 
 built-in AA with (for example) the hashmap from 
 https://github.com/economicmodeling/containers in combination 
 with a thread-local allocator.

 Here's an example of how to wrap an AA into a moderately safe 
 accessor. In the following example I create an additional 
 thread which concurrently adds odd numbers into the AA, while 
 the main threads add even nubmers:

 http://dpaste.dzfl.pl/06025e6374eb

 It segfaults on DPaste because you can't create threads there, 
 however it works ok on my machine. The output look like this:
 "0" : 0.140450140112896
 "1" : 1.140450129700608
 "2" : 2.140450140112896
 "3" : 3.140450129700608
 "4" : 4.140450140112896
 "5" : 5.140450129700608
 "6" : 6.140450140112896
 "7" : 7.140450129700608
 ...


Thanks a million. In my case, I would no longer manipulate the 
assoc array after creating and filling it. It would be read only 
(sorry, if I didn't make that clear in my initial post).

If I did manipulate it, however, I would either opt for something 
like https://github.com/economicmodeling/containers or opt for a 
different solution altogether.
Mar 01 2016