www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Problem with clear on shared associative array?

reply Andy Valencia <dont spam.me> writes:
The following code fails to compile; it appears from the error 
message that the library's clear() function is not ready to act 
on a shared AA?

synchronized class F {

private:
     string[int] mydict;

public:
     void clear() {
         this.mydict.clear();
     }
}

void
main()
{
     auto f = new shared F();
     f.clear();
}
May 26
parent reply Jonathan M Davis <newsgroup.d jmdavisprog.com> writes:
On Sunday, May 26, 2024 8:39:53 AM MDT Andy Valencia via Digitalmars-d-learn 
wrote:

 The following code fails to compile; it appears from the error
 message that the library's clear() function is not ready to act
 on a shared AA?

 synchronized class F {

 private:
      string[int] mydict;

 public:
      void clear() {
          this.mydict.clear();
      }
 }

 void
 main()
 {
      auto f = new shared F();
      f.clear();
 }


No operation on an associative array is thread-safe. As such, you should not
be doing _any_ operation on a shared AA without first locking a mutex to
protect it. Then you need to cast away shared to access or mutate it or do
whatever it is you want to do with it other than pass it around. And then
when you're done, you make sure that no thread-local references to the AA
exist, and you release the mutex. This is true for any type which is not
specifically designed to be shared, and none of the built-in types are
designed to be shared. They will work as shared so long as you protect them
appropriately and then temporarily cast away shared to operate on them as
thread-local, but they should never be mutated as shared, and it's on you to
make sure that the object is actually protected appropriately when you cast
away shared, since the language has no way of knowing when that's actually
safe to do. It can just prevent you from accidentally accessing an object
when it's shared, since when it's shared, the type system marks it as shared
across threads and thus not thread-safe to access.

The language is supposed to prevent you from doing any operations on a
shared object which are not guaranteed to be thread-safe (which is pretty
much anything other than passing it around by reference or pointer), but
unfortunately, since that wasn't entirely implemented up front, some of
those checks are currently behind the -preview=nosharedaccess flag and will
be made the default at some point in the future (but not now). So, some
shared operations may work when they really shouldn't, but the compiler will
prevent you from calling a function like clear, because it operates on
thread-local variables, not shared ones.

And with regards to AAs, you will need to be particularly careful with the
in operator, because it returns a pointer a value in the AA. So, if you cast
away shared to operate on it and then use in, you'll get a thread-local
pointer to the AA, meaning that the lock needs to stay in place as long as
that pointer is around, whereas a function like clear is done as soon as its
been called, so unless you're doing more with the AA at that point, you
could release the lock then.

Using synchronized is one way to get a mutex, and a synchronized function is
implicitly shared, so your clear function is both synchronized and shared,
meaning that you can call it on a shared object, and it will lock a mutex
when it's called, but you still need to cast away shared from mydict to be
able to call anything on it. The compiler is not smart enough to know that
removing shared is thread-safe within your function, so it will not do it
for you, and the clear function for AAs can't work on shared AAs, because it
has no way of guaranteeing that that's thread-safe. So, the solution is that
you must explicitly removed shared temporarily to call clear when you know
that it's thread-safe to do so.

- Jonathan M Davis
May 26
parent reply Andy Valencia <dont spam.me> writes:
On Sunday, 26 May 2024 at 20:00:50 UTC, Jonathan M Davis wrote:

 No operation on an associative array is thread-safe. As such, 
 you should not be doing _any_ operation on a shared AA without 
 first locking a mutex to protect it. Then you need to cast away 
 shared to access or mutate it or do whatever it is you want to 
 do with it other than pass it around. And then when you're 
 done, you make sure that no thread-local references to the AA 
 exist, and you release the mutex.
 ...


Thank you, that's exactly the big picture explanation I was 
hoping for.

For others wrestling with this issue, I found out how to cast to 
unshared at this article:

https://forum.dlang.org/thread/jwasqvrvkpqzimlutgrm forum.dlang.org

Andy
May 26
parent reply Serg Gini <kornburn yandex.ru> writes:
On Sunday, 26 May 2024 at 20:15:54 UTC, Andy Valencia wrote:

 For others wrestling with this issue, I found out how to cast 
 to unshared at this article:


You can check also this solution 
https://github.com/DmitryOlshansky/sharded-map
May 26
parent reply mw <mingwu gmail.com> writes:
On Monday, 27 May 2024 at 00:43:47 UTC, Serg Gini wrote:

 On Sunday, 26 May 2024 at 20:15:54 UTC, Andy Valencia wrote:

 For others wrestling with this issue, I found out how to cast 
 to unshared at this article:


 You can check also this solution 
 https://github.com/DmitryOlshansky/sharded-map



Pls NOTE: it is
a   `sharded` (meaning trunk-ed) NON-concurrent map,
not `shared` concurrent map.

These two words looks similar, but the meaning is very different.
May 26
parent reply Andy Valencia <dont spam.me> writes:
On Monday, 27 May 2024 at 04:04:03 UTC, mw wrote:

 Pls NOTE: it is
 a   `sharded` (meaning trunk-ed) NON-concurrent map,
 not `shared` concurrent map.


Assuming I put it in shared memory, in what way is it not able to 
be used concurrently?  It seems to have the needed lock 
operations?

Thanks,
Andy
May 27
parent "Richard (Rikki) Andrew Cattermole" <richard cattermole.co.nz> writes:
On 28/05/2024 12:36 PM, Andy Valencia wrote:

 On Monday, 27 May 2024 at 04:04:03 UTC, mw wrote:

 Pls NOTE: it is
 a   `sharded` (meaning trunk-ed) NON-concurrent map,
 not `shared` concurrent map.


 
 Assuming I put it in shared memory, in what way is it not able to be 
 used concurrently?  It seems to have the needed lock operations?
 
 Thanks,
 Andy


A concurrent data structure handles all of this for you.

AA's are not concurrent because it doesn't offer any protection.

Protecting a data structure with a mutex doesn't make it concurrent, but 
it may allow you to use it concurrently safely. Subtle difference!
May 27