www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - shared Mutex?

reply cy <dlang verge.info.tm> writes:
I was thinking of using threads in a D program (ignores unearthly 
wailing) and I need 1 thread for each unique string resource 
(database connection info). So I did this:

shared BackgroundDB[string] back;

I don't see any way to make less data shared there. If it weren't 
shared, it would be thread local, and two application threads 
trying to look up the same database would end up firing off two 
BackgroundDB threads, since they had separate copies of "back" 
that could not share keys. So it pretty much has to be shared. 
But that means freaking /everything/ has to be shared.

In the dedicated thread, I had it repeatedly waiting on a 
condition, and once that condition is signaled, it removes what's 
been queued up, and processes those queued items in the database. 
Except for one problem... conditions can't be shared.


 Error: non-shared method core.sync.condition.Condition.mutex is 
 not callable using a shared object


Obviously you shouldn't need mutexes if you're using shared... 
but how do you do conditions, then?

When I do something like this:

struct BackgroundDB {
   Condition stuff_ready;
   ...
}

Condition is implicitly converted to shared(Condition) when I 
create a shared(BackgroundDB), and BackgroundDB is implicitly 
converted to shared(BackgroundDB) when I have a shared 
BackgroundDB[string]. But shared(Condition) then has a 
shared(Mutex) inside it, and that can't be locked, since 
Mutex.lock is a non-shared function.

Is core.sync.mutex.Mutex even usable in D anymore? It seems every 
mutex that wasn't shared would be part of thread local data, so 
two threads locking on the same mutex would actually be locking 
separate mutexes.
Jun 09 2016
next sibling parent reply tcak <1ltkrs+3wyh1ow7kzn1k sharklasers.com> writes:
On Thursday, 9 June 2016 at 18:31:16 UTC, cy wrote:

 I was thinking of using threads in a D program (ignores 
 unearthly wailing) and I need 1 thread for each unique string 
 resource (database connection info). So I did this:

 shared BackgroundDB[string] back;

 I don't see any way to make less data shared there. If it 
 weren't shared, it would be thread local, and two application 
 threads trying to look up the same database would end up firing 
 off two BackgroundDB threads, since they had separate copies of 
 "back" that could not share keys. So it pretty much has to be 
 shared. But that means freaking /everything/ has to be shared.

 In the dedicated thread, I had it repeatedly waiting on a 
 condition, and once that condition is signaled, it removes 
 what's been queued up, and processes those queued items in the 
 database. Except for one problem... conditions can't be shared.


 Error: non-shared method core.sync.condition.Condition.mutex 
 is not callable using a shared object


 Obviously you shouldn't need mutexes if you're using shared... 
 but how do you do conditions, then?

 When I do something like this:

 struct BackgroundDB {
   Condition stuff_ready;
   ...
 }

 Condition is implicitly converted to shared(Condition) when I 
 create a shared(BackgroundDB), and BackgroundDB is implicitly 
 converted to shared(BackgroundDB) when I have a shared 
 BackgroundDB[string]. But shared(Condition) then has a 
 shared(Mutex) inside it, and that can't be locked, since 
 Mutex.lock is a non-shared function.

 Is core.sync.mutex.Mutex even usable in D anymore? It seems 
 every mutex that wasn't shared would be part of thread local 
 data, so two threads locking on the same mutex would actually 
 be locking separate mutexes.


Mutex, Condition, and Thread classes should be defined as shared 
as you experience, but they are not unfortunately. What you need 
to do is the define them as shared, and while calling their 
method, remove shared from them. Example is below:

class MyClass{
     private core.sync.mutex.Mutex mx;

     public this() shared{
         mx = cast(shared)( new core.sync.mutex.Mutex() );

         (cast()mx).lock();

         ... etc.
     }
}
Jun 09 2016
parent reply cy <dlang verge.info.tm> writes:
On Thursday, 9 June 2016 at 20:53:38 UTC, tcak wrote:


         (cast()mx).lock();


I was told casting away shared when there are still references to 
it is a bad idea. Like, the Mutex object might get corrupted if 
the garbage collector tries to move it while another thread is 
using it.

So thread 1 casts it to unshared, locks it, then allocates 
memory, triggering the GC to move things around. Meanwhile thread 
2 casts it to unshared, tries to lock it, and when it checks the 
locked bit, that area of memory has been replaced with another 
object entirely by the GC.

That seems like a really contrived situation, and maybe not a 
problem at all, if casting away shared doesn't make that memory 
eligible for being moved around. But, color me cautious before 
doing exactly what the FAQ says not to do.

https://dlang.org/faq.html#casting_from_shared
Jun 09 2016
parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/9/16 9:19 PM, cy wrote:

 On Thursday, 9 June 2016 at 20:53:38 UTC, tcak wrote:


         (cast()mx).lock();


 I was told casting away shared when there are still references to it is
 a bad idea. Like, the Mutex object might get corrupted if the garbage
 collector tries to move it while another thread is using it.


No, the GC doesn't care about shared in almost all circumstances, and 
certainly will not do anything different based on a cast.


 https://dlang.org/faq.html#casting_from_shared


That is if you are going to keep it unshared. Casting away shared 
temporarily is almost a requirement, as nobody writes shared-aware 
functions for types.

-Steve
Jun 10 2016
prev sibling parent Steven Schveighoffer <schveiguy yahoo.com> writes:
On 6/9/16 2:31 PM, cy wrote:


 Is core.sync.mutex.Mutex even usable in D anymore? It seems every mutex
 that wasn't shared would be part of thread local data, so two threads
 locking on the same mutex would actually be locking separate mutexes.


Yes, but this is because Mutex existed way before shared did.

IMO Mutex should be ALWAYS shared. As you suggest, an unshared mutex is 
a useless thing.

You can have data that is shared, but not actually typed as shared, 
which is what Mutex traditionally works with.

This is a bug that needs to be fixed -- everything in core.sync should 
work with shared and unshared types. And then we should eventually get 
rid of (or turn into no-ops) the unshared members.

-Steve
Jun 09 2016