www.digitalmars.com         C & C++   DMDScript  

digitalmars.D - Should core.sync.mutex.Mutex, core.sync.condition.Condition,

reply Jonathan M Davis <jmdavisProg gmx.com> writes:
I don't know if we can answer this for sure at the moment given the ongoing 
discussion on shared, but looking at core.sync, it occurred to me that there's 
a major problem with the classes in there. None of the work with shared. And 
unless I'm missing something here, I don't see how many of them are even 
useful as anything other than shared. After all, what good is a mutex which is 
thread-local? But none of the methods on Mutex or its friends are shared.

So, the question is should the all have their methods shared? And if they 
should, is there any reason to have non-shared overloads for them? What good 
are they as anything other than shared? How is anyone using them right now?

- Jonathan M Davis
Nov 17 2012
next sibling parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 18-11-2012 05:46, Jonathan M Davis wrote:

 I don't know if we can answer this for sure at the moment given the ongoing
 discussion on shared, but looking at core.sync, it occurred to me that there's
 a major problem with the classes in there. None of the work with shared. And
 unless I'm missing something here, I don't see how many of them are even
 useful as anything other than shared. After all, what good is a mutex which is
 thread-local? But none of the methods on Mutex or its friends are shared.

 So, the question is should the all have their methods shared? And if they
 should, is there any reason to have non-shared overloads for them? What good
 are they as anything other than shared? How is anyone using them right now?

 - Jonathan M Davis


Not at this point in time. It would break a ridiculous amount of code if 
we did this, given the current extremely annoying nature of shared.

Most D code I have seen in the wild just shares mutexes, conditions, etc 
with __gshared or some other mechanism anyway, so I don't think there's 
anything to gain. Like, what would shared actually buy you here?

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org
Nov 17 2012
parent reply Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 18, 2012 05:51:00 Alex R=C3=B8nne Petersen wrote:

 On 18-11-2012 05:46, Jonathan M Davis wrote:

 I don't know if we can answer this for sure at the moment given the=





 ongoing
 discussion on shared, but looking at core.sync, it occurred to me t=




hat

 there's a major problem with the classes in there. None of the work=




 with

 shared. And unless I'm missing something here, I don't see how many=




 of

 them are even useful as anything other than shared. After all, what=




 good

 is a mutex which is thread-local? But none of the methods on Mutex =




or its

 friends are shared.
=20
 So, the question is should the all have their methods shared? And i=




f they

 should, is there any reason to have non-shared overloads for them? =




What

 good are they as anything other than shared? How is anyone using th=




em

 right now?
=20
 - Jonathan M Davis


=20
 Not at this point in time. It would break a ridiculous amount of code=


 if

 we did this, given the current extremely annoying nature of shared.
=20
 Most D code I have seen in the wild just shares mutexes, conditions, =


etc

 with __gshared or some other mechanism anyway, so I don't think there=


's

 anything to gain. Like, what would shared actually buy you here?


__gshared is a good reason for leaving non-shared overloads, but isn't =
code=20
really supposed to be using shared and not __gshared unless it's specif=
ically=20
extern(C)? That being the case, I'd expect shared to be the correct thi=
ng to=20
use with mutexes normally, and right now, that won't work without a ton=
 of=20
casting or adding shared overloads.

- Jonathan M Davis
Nov 17 2012
parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 18-11-2012 05:58, Jonathan M Davis wrote:

 On Sunday, November 18, 2012 05:51:00 Alex Rønne Petersen wrote:

 On 18-11-2012 05:46, Jonathan M Davis wrote:

 I don't know if we can answer this for sure at the moment given the
 ongoing
 discussion on shared, but looking at core.sync, it occurred to me that
 there's a major problem with the classes in there. None of the work with
 shared. And unless I'm missing something here, I don't see how many of
 them are even useful as anything other than shared. After all, what good
 is a mutex which is thread-local? But none of the methods on Mutex or its
 friends are shared.

 So, the question is should the all have their methods shared? And if they
 should, is there any reason to have non-shared overloads for them? What
 good are they as anything other than shared? How is anyone using them
 right now?

 - Jonathan M Davis


 Not at this point in time. It would break a ridiculous amount of code if
 we did this, given the current extremely annoying nature of shared.

 Most D code I have seen in the wild just shares mutexes, conditions, etc
 with __gshared or some other mechanism anyway, so I don't think there's
 anything to gain. Like, what would shared actually buy you here?


 __gshared is a good reason for leaving non-shared overloads, but isn't code
 really supposed to be using shared and not __gshared unless it's specifically
 extern(C)? That being the case, I'd expect shared to be the correct thing to
 use with mutexes normally, and right now, that won't work without a ton of
 casting or adding shared overloads.

 - Jonathan M Davis


I don't know what anything that has to do with shared is supposed or not 
supposed to do. It is not clear, even to the primary language designers, 
what shared actually is, so I really can't say anything to this...

All I can say is, let's wait with this until we know what shared is 
going to actually do. We don't want to do something we'll regret later.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org
Nov 17 2012
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
11/18/2012 11:13 AM, Alex Rønne Petersen пишет:

 On 18-11-2012 05:58, Jonathan M Davis wrote:

 On Sunday, November 18, 2012 05:51:00 Alex Rønne Petersen wrote:

 On 18-11-2012 05:46, Jonathan M Davis wrote:

 I don't know if we can answer this for sure at the moment given the
 ongoing
 discussion on shared, but looking at core.sync, it occurred to me that
 there's a major problem with the classes in there. None of the work
 with
 shared. And unless I'm missing something here, I don't see how many of
 them are even useful as anything other than shared. After all, what
 good
 is a mutex which is thread-local? But none of the methods on Mutex
 or its
 friends are shared.

 So, the question is should the all have their methods shared? And if
 they
 should, is there any reason to have non-shared overloads for them? What
 good are they as anything other than shared? How is anyone using them
 right now?

 - Jonathan M Davis


 Not at this point in time. It would break a ridiculous amount of code if
 we did this, given the current extremely annoying nature of shared.

 Most D code I have seen in the wild just shares mutexes, conditions, etc
 with __gshared or some other mechanism anyway, so I don't think there's
 anything to gain. Like, what would shared actually buy you here?


 __gshared is a good reason for leaving non-shared overloads, but isn't
 code
 really supposed to be using shared and not __gshared unless it's
 specifically
 extern(C)? That being the case, I'd expect shared to be the correct
 thing to
 use with mutexes normally, and right now, that won't work without a
 ton of
 casting or adding shared overloads.

 - Jonathan M Davis


 I don't know what anything that has to do with shared is supposed or not
 supposed to do. It is not clear, even to the primary language designers,
 what shared actually is, so I really can't say anything to this...


Let's please stop this mantra. It buy us nothing but FUD.

One thing is clear - shared is not going away over night. Yes, having 
auto-magic atomic ops for various builtins is discussed, the kind of 
memory model is discussed.

Everything else stays. It's the globally shared data and it has tight 
restrictions on what you can do with it. As such it is (just like 
immutable) is safely passable between threads.


 All I can say is, let's wait with this until we know what shared is
 going to actually do. We don't want to do something we'll regret later.


Obviously we know what shared is. It's just certain details need 
straightening up. And certainly druntime/std.concurrency should add more 
support for it where meaningful.

-- 
Dmitry Olshansky
Nov 18 2012
parent reply =?UTF-8?B?QWxleCBSw7hubmUgUGV0ZXJzZW4=?= <alex lycus.org> writes:
On 18-11-2012 10:58, Dmitry Olshansky wrote:

 11/18/2012 11:13 AM, Alex Rønne Petersen пишет:

 On 18-11-2012 05:58, Jonathan M Davis wrote:

 On Sunday, November 18, 2012 05:51:00 Alex Rønne Petersen wrote:

 On 18-11-2012 05:46, Jonathan M Davis wrote:

 I don't know if we can answer this for sure at the moment given the
 ongoing
 discussion on shared, but looking at core.sync, it occurred to me that
 there's a major problem with the classes in there. None of the work
 with
 shared. And unless I'm missing something here, I don't see how many of
 them are even useful as anything other than shared. After all, what
 good
 is a mutex which is thread-local? But none of the methods on Mutex
 or its
 friends are shared.

 So, the question is should the all have their methods shared? And if
 they
 should, is there any reason to have non-shared overloads for them?
 What
 good are they as anything other than shared? How is anyone using them
 right now?

 - Jonathan M Davis


 Not at this point in time. It would break a ridiculous amount of
 code if
 we did this, given the current extremely annoying nature of shared.

 Most D code I have seen in the wild just shares mutexes, conditions,
 etc
 with __gshared or some other mechanism anyway, so I don't think there's
 anything to gain. Like, what would shared actually buy you here?


 __gshared is a good reason for leaving non-shared overloads, but isn't
 code
 really supposed to be using shared and not __gshared unless it's
 specifically
 extern(C)? That being the case, I'd expect shared to be the correct
 thing to
 use with mutexes normally, and right now, that won't work without a
 ton of
 casting or adding shared overloads.

 - Jonathan M Davis


 I don't know what anything that has to do with shared is supposed or not
 supposed to do. It is not clear, even to the primary language designers,
 what shared actually is, so I really can't say anything to this...


 Let's please stop this mantra. It buy us nothing but FUD.

 One thing is clear - shared is not going away over night. Yes, having
 auto-magic atomic ops for various builtins is discussed, the kind of
 memory model is discussed.

 Everything else stays. It's the globally shared data and it has tight
 restrictions on what you can do with it. As such it is (just like
 immutable) is safely passable between threads.


There have been proposals to a completely different meaning of shared 
that has nothing to do with memory models or atomic ops. There is no FUD 
here. Shoehorning shared in its current state into core.sync is just 
plain unreasonable given that nothing has been set in stone about the 
meaning of shared at all yet.


 All I can say is, let's wait with this until we know what shared is
 going to actually do. We don't want to do something we'll regret later.


 Obviously we know what shared is. It's just certain details need
 straightening up. And certainly druntime/std.concurrency should add more
 support for it where meaningful.


We don't know at all what shared is. Have you been following the other 
thread? People have wildly different ideas of what it should do.



I don't see why we should be in a hurry to add shared to core.sync in 
particular. Practically all of the runtime and standard library can't 
work properly with shared anyway, so it's not going to change a whole 
lot. I think it's much more sensible to wait until shared's semantics 
are set in stone and *then* start adapting the runtime and standard library.

I can't stop anyone from just doing this, of course, but it seems like a 
premature change to me.

-- 
Alex Rønne Petersen
alex lycus.org
http://lycus.org
Nov 18 2012
parent reply Dmitry Olshansky <dmitry.olsh gmail.com> writes:
11/18/2012 10:36 PM, Alex Rønne Petersen пишет:
[snip]

 Everything else stays. It's the globally shared data and it has tight
 restrictions on what you can do with it. As such it is (just like
 immutable) is safely passable between threads.


 There have been proposals to a completely different meaning of shared
 that has nothing to do with memory models or atomic ops. There is no FUD
 here. Shoehorning shared in its current state into core.sync is just
 plain unreasonable given that nothing has been set in stone about the
 meaning of shared at all yet.


I'd argue that shared on these objects has nothing to do with memory 
models and atomics. It's about global visibility and access to API that 
is interlocked on OS-level anyway. It works with any sane definition 
that doesn't forget to add that 'shared' implies global visibility.


 All I can say is, let's wait with this until we know what shared is
 going to actually do. We don't want to do something we'll regret later.


 Obviously we know what shared is. It's just certain details need
 straightening up. And certainly druntime/std.concurrency should add more
 support for it where meaningful.


 We don't know at all what shared is. Have you been following the other
 thread? People have wildly different ideas of what it should do.


I admit dismissing a lot of proposals from that thread on sight. The 
ones being about shared somehow 'just working' and getting you safe way 
to sharing and the one about auto-magically attaching mutexes to shared 
stuff.

Given now that the thread is mostly spent. If there many good different 
concepts I'd appreciate if you list these here.


 I don't see why we should be in a hurry to add shared to core.sync in
 particular.



 Practically all of the runtime and standard library can't
 work properly with shared anyway, so it's not going to change a whole
 lot.


In part because even things that are supposed to always be shared in 
fact are not working as shared. And that's because nobody have found the 
time to go on and do this work.

Add the fact that say class instances have monitor field when everything 
is TLS by default for me is indication of one thing only - during a move 
to TLS by default and shared a lot of things were not updated 
accordingly because of the lack of time and other issues.


 I think it's much more sensible to wait until shared's semantics
 are set in stone and *then* start adapting the runtime and standard
 library.

 I can't stop anyone from just doing this, of course, but it seems like a
 premature change to me.


What change in semantics of shared could stop mutex object from making 
sense only as a shared object? I mean the kind of change that can 
realistically can happen not some vague idea.

-- 
Dmitry Olshansky
Nov 18 2012
next sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 18, 2012 23:28:03 Dmitry Olshansky wrote:

 What change in semantics of shared could stop mutex object from making
 sense only as a shared object? I mean the kind of change that can
 realistically can happen not some vague idea.


I thought that it was blindingly clear from TDPL and even the online spec what 
shared is for. It marks variables as being shared across threads rather than 
thread local. I don't see how that could possibly be changing. __gshared isn't 
even _mentioned_ in TDPL, and I think that it gets used as much as it does is 
entirely because of some of the current issues with shared's implementation in 
the compiler and the lack of support in druntime and Phobos.

There are definitely questions as to what exactly should happen when a variable 
is marked as shared, but they have to do with what protections you get and 
what conversions are possible. They still don't fundamentally change the fact 
that the point of shared is to share an instance across threads rather than 
have it be thread local like everything is by default.

So, I concur in that I don't see how the semantics of shared could possibly 
not be appropriate for mutexes. I started this thread primarily because I was 
shocked that Mutex, Condition, etc. weren't marked with shared already, and I 
didn't see how they could even be used without it (and apparently, the answer 
to that is that almost everyone cops out and uses __gshared).

- Jonathan M Davis
Nov 18 2012
prev sibling next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
To separate an important notion I'm trying to express.

I assume that:

class X{
...
	shared foo(...);
	void bar(...);
...
}

shared X sharedStuff;
X localStuff;

Implies under any viable definition of shared that (including the 
current one and any pending clarifications):

a) foo can be called with sharedStuff in multiple threads at the same 
time. Thus it has to be thread safe (irregardless of possible 
 system/ trusted hackery inside).

b) localStuff on the other hand can called with  bar (and any non-shared 
methods) only in a single thread at any time. If localStuff is defined 
at global scope each thread has its own copy thus this requirement is 
fulfilled kind of automatically.

The major benefits of shared lie in cleaning up the opposite - unshared 
stuff. Having shared enables us to archieve great things we haven't 
looked at yet (maybe because of bugs/issues in the other departments). 
I'll name a few.

We can have lock-less and _safe_ file I/O  by default unlike e.g. C and 
pretty much everybody else. I should have said safe & _faster_  by 
default. This is why I believe D should stop relying on C's I/O as 
anything but fallback. We even had a partially complete design for new 
std.io that does it.

Same goes for memory allocations - lock-less thread-local allocators are 
possible _safely_ and by default (though it would have to go to the OS 
from time to time and likely to lock something here). It's not explored 
and remains as unused potential.

-- 
Dmitry Olshansky
Nov 18 2012
prev sibling parent Sean Kelly <sean invisibleduck.org> writes:
On Nov 18, 2012, at 12:44 PM, Jonathan M Davis <jmdavisProg gmx.com> =
wrote:

=20
 So, I concur in that I don't see how the semantics of shared could =


possibly=20

 not be appropriate for mutexes. I started this thread primarily =


because I was=20

 shocked that Mutex, Condition, etc. weren't marked with shared =


already, and I=20

 didn't see how they could even be used without it (and apparently, the =


answer=20

 to that is that almost everyone cops out and uses __gshared).


I tried this once and it cascaded to requiring modifications of various =
definitions on core.sys.posix to add a "shared" qualifier, and since I =
wasn't ready to do that I rolled back the changes.  I guess the =
alternative would be to have a shared equivalent for every operation =
that basically just casts away shared and then calls the non-shared =
function, but that's such a terrible design I've been resisting it.=
Nov 18 2012
prev sibling next sibling parent Dmitry Olshansky <dmitry.olsh gmail.com> writes:
11/18/2012 8:46 AM, Jonathan M Davis пишет:

 I don't know if we can answer this for sure at the moment given the ongoing
 discussion on shared, but looking at core.sync, it occurred to me that there's
 a major problem with the classes in there. None of the work with shared. And
 unless I'm missing something here, I don't see how many of them are even
 useful as anything other than shared. After all, what good is a mutex which is
 thread-local? But none of the methods on Mutex or its friends are shared.


None of them are useful without shared.They should work with shared, but 
that'll break code outright. I think 2 overloads per method with TLS 
versions going through the deprecation path.


 So, the question is should the all have their methods shared? And if they
 should, is there any reason to have non-shared overloads for them? What good
 are they as anything other than shared? How is anyone using them right now?


I do suspect that most people use _gshared with them as it is the less 
painful way. Another way is to have them shared and cast them to TLS as 
needed. Since mutex, condition etc. are opaque types around OS 
primitives TLS doesn't make sense at all but currently allows to call 
their methods.

-- 
Dmitry Olshansky
Nov 18 2012
prev sibling parent Jonathan M Davis <jmdavisProg gmx.com> writes:
On Sunday, November 18, 2012 14:48:02 Sean Kelly wrote:

 On Nov 18, 2012, at 12:44 PM, Jonathan M Davis <jmdavisProg gmx.com> wrote:

 So, I concur in that I don't see how the semantics of shared could
 possibly
 not be appropriate for mutexes. I started this thread primarily because I
 was shocked that Mutex, Condition, etc. weren't marked with shared
 already, and I didn't see how they could even be used without it (and
 apparently, the answer to that is that almost everyone cops out and uses
 __gshared).


 
 I tried this once and it cascaded to requiring modifications of various
 definitions on core.sys.posix to add a "shared" qualifier, and since I
 wasn't ready to do that I rolled back the changes.  I guess the alternative
 would be to have a shared equivalent for every operation that basically
 just casts away shared and then calls the non-shared function, but that's
 such a terrible design I've been resisting it.


Well, this is certainly going to need to be sorted out. And given that it's 
not unreasonable that both shared and __gshared be used with the type is 
core.sync, I don't think we have much choice to duplicate most of those 
functions (probably using casting internally to avoid actually duplicating the 
implementation in most cases) unless __gshared becomes implicitly convertible 
to shared.

- Jonathan M Davis
Nov 20 2012