digitalmars.D - D 2.0 FAQ on `shared`
- Marco Leise (45/69) Oct 20 2014 What if I have a thread that contains some shared data? Should
- Sean Kelly (95/170) Oct 20 2014 Since Thread is by its very nature a shared thing, Thread should
- Marco Leise (85/200) Oct 21 2014 =20
- Sean Kelly (61/170) Oct 21 2014 Good point about a shared class not having any unshared methods.
- "Marc =?UTF-8?B?U2Now7x0eiI=?= <schuetzm gmx.net> (11/18) Oct 21 2014 `shared` applies to the implicit `this` parameter, not to the
- Sean Kelly (31/37) Oct 21 2014 I think "synchronized" should work basically the same way. For
- Marco Leise (19/106) Oct 21 2014 I have nothing to add.=20
- Sean Kelly (12/20) Oct 21 2014 No. But the issue in general concerns me. Like say my class
- Marco Leise (5/17) Oct 21 2014 No no, a FILE* is obviously immutable.
What guarantees is shared supposed to provide? Shared means that multiple threads can access the data. The guarantee is that if it is not shared, and not immutable, that only the current thread can see it.What if I have a thread that contains some shared data? Should the thread be created as shared, be cast to shared after construction or not be shared and fine grained shared applied to the respective shared data fields?What does shared have to do with synchronization? Only shared data can be synchronized. It makes no sense to synchronize thread local data.Define synchronized. With atomic ops on word size items this is clear, but what does it mean for aggregates? The language shows us no connection between synchronization and the shared data. What is one unit of data that is to be synchronized?What does shared have to do with memory barriers? Reading/writing shared data emits memory barriers to ensure sequential consistency (not implemented).That's exactly the problem. It assumes the unit is a word size item. If I have a Mutex to protect my unit of shared data, I don't need "volatile" handling of shared data. private shared class SomeThread : Thread { private: Condition m_condition; bool m_shutdown = false; ... } m_shutdown will be shared and it is shared data, but it is synchronized by the Mutex contained in that condition. Automatic memory barriers and such would only slow down execution.What are the semantics of casting FROM unshared TO shared? Make sure there are no other unshared references to that same data. What are the semantics of casting FROM shared TO unshared? Make sure there are no other shared references to that same data.That's just wrong to ask. `SomeThread` is a worker thread and data is passed to it regularly through a shared reference that is certainly never going away until the thread dies. Yet I must be able to "unshare" it's list of work items to process them. Now let's say I have an "empty" property. Shared or unshared? override property bool empty() const { return m_list.empty; } It is only called internally by the thread itself after entering a certain critical section. I _know_ that m_list wont be accessible by other threads while .empty is running. But this seeming 1:1 relationship between entering "the" critical section and stripping shared is of course non-existent. Aggregates may contain Mutexes protecting different fields or even stacking on top of each other. So the text should read:What are the semantics of casting FROM shared TO unshared? Make sure that during the period the data is unshared, no other thread can modify those parts of it that you will be accessing. If you don't use synchronization objects with built-in memory-barriers like a Mutex, it is your responsibility to properly synchronize data access through e.g. atomicLoad/Store.That at least in general sanctifies casting away shared for the purpose of calling a method under protection of a user defined critical section. -- Marco
Oct 20 2014
On Monday, 20 October 2014 at 13:29:47 UTC, Marco Leise wrote:Since Thread is by its very nature a shared thing, Thread should probably be defined as shared. But more generally it depends on the use case.What guarantees is shared supposed to provide? Shared means that multiple threads can access the data. The guarantee is that if it is not shared, and not immutable, that only the current thread can see it.What if I have a thread that contains some shared data? Should the thread be created as shared, be cast to shared after construction or not be shared and fine grained shared applied to the respective shared data fields?I think there's some conflation of two separate uses of "synchronized" here. I think the above is actually talking about synchronized methods (ie. involving a mutex).What does shared have to do with synchronization? Only shared data can be synchronized. It makes no sense to synchronize thread local data.Define synchronized. With atomic ops on word size items this is clear, but what does it mean for aggregates? The language shows us no connection between synchronization and the shared data. What is one unit of data that is to be synchronized?I'd say the real problem is more that it assumes, or at least suggests, that sequential consistency of shared variables will result in a correct program. It won't, for any non-trivial uses of shared variables. Lock-free programming is really, really hard, even for experts. Using shared variables in this way shouldn't be easy semantically because it provides a false sense of security, resulting in programs that are silently broken in weird ways under some conditions but not others.What does shared have to do with memory barriers? Reading/writing shared data emits memory barriers to ensure sequential consistency (not implemented).That's exactly the problem. It assumes the unit is a word size item.If I have a Mutex to protect my unit of shared data, I don't need "volatile" handling of shared data. private shared class SomeThread : Thread { private: Condition m_condition; bool m_shutdown = false; ... }Yep. This is one of my biggest issues with shared as it applies to user-defined types. I even raised it in the now defunct concurrency mailing list before the design was finalized. Sadly, there's no good way to sort this out, because: shared class A { int m_count = 0; void increment() shared { m_count.atomicOp!"+="(1); } int getCount() synchronized { return m_count; } } If we make accesses of shared variables non-atomic inside synchronized methods, there may be conflicts with their use in shared methods. Also: shared class A { void doSomething() synchronized { doSomethingElse(); } private void doSomethingElse() synchronized { } } doSomethingElse must be synchronized even if I as a programmer know it doesn't have to be because the compiler insists it must be. And I know that private methods are visible within the module, but the same rule applies. In essence, we can't avoid recursive mutexes for implementing synchronized, and we're stuck with a lot of recursive locks and unlocks no matter what, as soon as we slap a "shared" label on something.m_shutdown will be shared and it is shared data, but it is synchronized by the Mutex contained in that condition. Automatic memory barriers and such would only slow down execution.Yes. Though there's no overhead for having a Mutex synchronize one more operation. A Mutex is basically just a shared variable indicating locked state. When you leave a Mutex a shared variable is written to to indicate that the Mutex is unlocked, and the memory model in that language/platform/cpu guarantees that all operations logically occurring before that shared write actually do complete before the shared write, at least to anyone who acquires that same mutex before looking at the protected data (ie. there's a reader-writer contract).Sure, but at that point they are no longer referenced by the shared Thread, correct? The rule is simply that you can't be trying to read or write data using both shared and unshared operations, because of that reader-writer contract I mentioned above.What are the semantics of casting FROM unshared TO shared? Make sure there are no other unshared references to that same data. What are the semantics of casting FROM shared TO unshared? Make sure there are no other shared references to that same data.That's just wrong to ask. `SomeThread` is a worker thread and data is passed to it regularly through a shared reference that is certainly never going away until the thread dies. Yet I must be able to "unshare" it's list of work items to process them.Now let's say I have an "empty" property. Shared or unshared? override property bool empty() const { return m_list.empty; } It is only called internally by the thread itself after entering a certain critical section. I _know_ that m_list wont be accessible by other threads while .empty is running.So one thing about shared that Walter confirmed at some point is that atomic ops won't be imposed on operations within a shared method. But I suspect that someone is likely to read the preceding sentence and say "woah! We never said that! And if we did, that's wrong!" In short, I agree with you that shared, as described, kind of sucks here because you're stuck with a ton of inefficiency that you, as an intelligent programmer, know is unnecessary.But this seeming 1:1 relationship between entering "the" critical section and stripping shared is of course non-existent. Aggregates may contain Mutexes protecting different fields or even stacking on top of each other. So the text should read:It's more complicated than that, because you don't know how long a given operation needs to propagate to another CPU. Simply performing a shared write is meaningless if something else is performing an unshared read because the optimization happens at both points--the write side and the read side. In essence, the CPU performs the same optimizations as a compiler. Depending on the architecture, reads may be rearranged to occur before other reads or writes, and writes may be rearranged to occur before other reads and writes. On most architectures the CPU makes some intelligent guesses about what operations are safe to rearrange (look into "dependent loads"), though on some few others like the DEC Alpha (a CPU invented by crazy people), they do not and if you don't explicitly tell them what needs to happen, it won't. Basically what's needed is some way to have the compiler optimize according to the same rules as the CPU (the goal of "shared"). Or in lieu of that, to have some "don't optimize this" instruction to tell the compiler to keep it's dirty hands off your carefully constructed code so the only thing you need to worry about is what the CPU is trying to do. This is what "volatile" was meant for in D1 and I really liked it, but I think I was the only one. There's a paper on release consistency that I think is fantastic. I'll link it later if I can find it on the interweb. CPUs seem to be converging on even more strict memory ordering than release consistency, but the release consistency model is really fantastic as it's basically equivalent to how mutexes work and so it's a model everyone already understands.What are the semantics of casting FROM shared TO unshared? Make sure that during the period the data is unshared, no other thread can modify those parts of it that you will be accessing. If you don't use synchronization objects with built-in memory-barriers like a Mutex, it is your responsibility to properly synchronize data access through e.g. atomicLoad/Store.That at least in general sanctifies casting away shared for the purpose of calling a method under protection of a user defined critical section.
Oct 20 2014
Am Mon, 20 Oct 2014 16:18:51 +0000 schrieb "Sean Kelly" <sean invisibleduck.org>:On Monday, 20 October 2014 at 13:29:47 UTC, Marco Leise wrote:=20 In a single-threaded application in particular, there is an unshared thread :p But to the point: Doesn't defining it as shared means that it can not have _any_ unshared methods? Ok, fair enough. So even if a method is only working on technically unshared parts of the thread's data, it has to cast everything to unshared itself. This makes sense since `this`, the Thread itself is still shared.What if I have a thread that contains some shared data? Should the thread be created as shared, be cast to shared after construction or not be shared and fine grained shared applied to the respective shared data fields?=20 Since Thread is by its very nature a shared thing, Thread should=20 probably be defined as shared. But more generally it depends on=20 the use case.[=E2=80=A6]Well, when you talk about "shared and unshared operations" further down, I took it as the set of operations ensuring thread-safety over a particular set of shared data. That code above is just a broken set of such operations. I.e. in this case the programmer must decide between mutex synchronization and atomic read-modify-write. That's not too much to ask.If I have a Mutex to protect my unit of shared data, I don't need "volatile" handling of shared data. private shared class SomeThread : Thread { private: Condition m_condition; bool m_shutdown =3D false; ... }=20 Yep. This is one of my biggest issues with shared as it applies=20 to user-defined types. I even raised it in the now defunct=20 concurrency mailing list before the design was finalized. Sadly,=20 there's no good way to sort this out, because: =20 shared class A { int m_count =3D 0; void increment() shared { m_count.atomicOp!"+=3D"(1); } =20 int getCount() synchronized { return m_count; } } =20 If we make accesses of shared variables non-atomic inside=20 synchronized methods, there may be conflicts with their use in=20 shared methods. Also:shared class A { void doSomething() synchronized { doSomethingElse(); } =20 private void doSomethingElse() synchronized { =20 } } =20 doSomethingElse must be synchronized even if I as a programmer=20 know it doesn't have to be because the compiler insists it must=20 be. And I know that private methods are visible within the=20 module, but the same rule applies. In essence, we can't avoid=20 recursive mutexes for implementing synchronized, and we're stuck=20 with a lot of recursive locks and unlocks no matter what, as soon=20 as we slap a "shared" label on something.Imagine you have a shared root object that contains a deeply nested private data structure that is technically unshared. Then it becomes not only one more method of the root object that needs to be `synchronized` but it cascades all the way down its private fields as well. One ends up requiring data structures designed for single-threaded execution to grow synchronized methods over night even though they aren't _really_ used concurrently my multiple threads. =20The work items? They stay referenced by the shared Thread until it is done with them. In this particular implementation an item is moved from the list to a separate field that denotes the current item and then the Mutex is released. This current item is technically unshared now, because only this thread can really see it, but as far as the language is concerned there is a shared reference to it because shared applies transitively. The same goes for the list of items while it is under the Mutex protection.=20 Sure, but at that point they are no longer referenced by the=20 shared Thread, correct?What are the semantics of casting FROM shared TO unshared? Make sure there are no other shared references to that same data.That's just wrong to ask. `SomeThread` is a worker thread and data is passed to it regularly through a shared reference that is certainly never going away until the thread dies. Yet I must be able to "unshare" it's list of work items to process them.The rule is simply that you can't be=20 trying to read or write data using both shared and unshared=20 operations, because of that reader-writer contract I mentioned=20 above.Something along that line yes. The exact formulation may need to be ironed out, but what the FAQ says right now doesn't work. When you say (un)shared operations that maps to any means of ensuring thread-safe operations on a set of data. It can range from putting synchronized in front of your class to the exact order of executing a series of loads and stored in a lock-free algorithm. Anything between a single shared basic data type and full blown synchronized class is too complicated for the compiler to see through. So a simple definition of `shared` like the FAQ attempts wont fly. Most methods are "somewhat shared": private void workOnAnItem() shared { // m_current is technically never shared, // but we cannot describe this with `shared`. // Hence I manually unshare where appropriate. synchronized (m_condition.unshared.mutex) { m_current =3D m_list.unshared.front; m_list.unshared.removeFront(); } m_current.unshared.doSomthing(); } =20You are right, my point was that the original formulation is so strict that can only come from the point of view of using shared in message passing. It doesn't spend a thought on how a shared(Thread) is supposed to be both referable and able to unshare internal lists during processing.[=E2=80=A6] So the text should read:=20 It's more complicated than that, because you don't know how long=20 a given operation needs to propagate to another CPU. Simply=20 performing a shared write is meaningless if something else is=20 performing an unshared read because the optimization happens at=20 both points--the write side and the read side.What are the semantics of casting FROM shared TO unshared? Make sure that during the period the data is unshared, no other thread can modify those parts of it that you will be accessing. If you don't use synchronization objects with built-in memory-barriers like a Mutex, it is your responsibility to properly synchronize data access through e.g. atomicLoad/Store.That at least in general sanctifies casting away shared for the purpose of calling a method under protection of a user defined critical section.In essence, the CPU performs the same optimizations as a=20 compiler. [=E2=80=A6]Yeah, I know, except for the DEC Alpha part.Basically what's needed is some way to have the compiler optimize=20 according to the same rules as the CPU (the goal of "shared"). =20 Or in lieu of that, to have some "don't optimize this"=20 instruction to tell the compiler to keep it's dirty hands off=20 your carefully constructed code so the only thing you need to=20 worry about is what the CPU is trying to do. This is what=20 "volatile" was meant for in D1 and I really liked it, but I think=20 I was the only one.Count me in. Anecdotally I once tried to see if I can write a minimal typed malloc() that is faster than temalloc. It went way over budget from a single CAS instruction, where temalloc mostly works on thread local pools. These synchronizations stall the CPU _that_ much, that I don't see how someone writing lock-free algorithms with `shared` will accept implicit full barriers placed by the language. This is a dead-end to me. Mostly what I use is load-acquire and store-release, but sometimes raw atomic read access is sufficient as well. So ideally I would like to see: volatile -> compiler doesn't reorder stuff and on top of that: atomicLoad/Store -> CPU doesn't reorder stuff in the pipeline in the way I described by MemoryOrder.xxx A shared variable need not be volatile, but a volatile variable is implicitly shared.There's a paper on release consistency that I think is fantastic.=20 I'll link it later if I can find it on the interweb. CPUs seem=20 to be converging on even more strict memory ordering than release=20 consistency, but the release consistency model is really=20 fantastic as it's basically equivalent to how mutexes work and so=20 it's a model everyone already understands.--=20 Marco
Oct 21 2014
On Tuesday, 21 October 2014 at 13:10:57 UTC, Marco Leise wrote:Am Mon, 20 Oct 2014 16:18:51 +0000 schrieb "Sean Kelly" <sean invisibleduck.org>: But to the point: Doesn't defining it as shared means that it can not have _any_ unshared methods? Ok, fair enough. So even if a method is only working on technically unshared parts of the thread's data, it has to cast everything to unshared itself. This makes sense since `this`, the Thread itself is still shared.Good point about a shared class not having any unshared methods. I guess that almost entirely eliminates the cases where I might define a class as shared. For example, the MessageBox class in std.concurrency has one or two ostensibly shared methods and the rest are unshared. And it's expected for there to be both shared and unshared references to the object held simultaneously. This is by design, and the implementation would either be horribly slow or straight-up broken if done another way. Also, of the shared methods that exist, there are synchronized blocks but they occur at a fine grain within the shared methods rather than the entire method being shared. I think that labeling entire methods as synchronized is an inherently flawed concept, as it contradicts the way mutexes are supposed to be used (which is to hold the lock for as short a time as possible). I hate to say it, but if I were to apply shared/synchronized labels to class methods it would simply be to service user requests rather than because I think it would actually make the code better or safer.I agree. I was being pedantic for the sake of informing anyone who wasn't aware. There are times where I have some fields be lock-free and others protected by a mutex though. See Thread.isRunning, for example. There are times where a write delay is acceptable and the possibility of tearing is irrelevant. But I think this falls pretty squarely into the "expert" category--I don't care if the language makes it easy.shared class A { int m_count = 0; void increment() shared { m_count.atomicOp!"+="(1); } int getCount() synchronized { return m_count; } } If we make accesses of shared variables non-atomic inside synchronized methods, there may be conflicts with their use in shared methods. Also:Well, when you talk about "shared and unshared operations" further down, I took it as the set of operations ensuring thread-safety over a particular set of shared data. That code above is just a broken set of such operations. I.e. in this case the programmer must decide between mutex synchronization and atomic read-modify-write. That's not too much to ask.I need to give it some more thought, but I think the way this should work is for shared to not be transitive, but for the compiler to require that non-local variables accessed within a shared method must either be declared as shared or the access must occur within a synchronized block. This does trust the programmer a bit more than the current design, but in exchange it encourages a programming model that actually makes sense. It doesn't account for the case where I'm calling pthread_mutex_lock on an unshared variable though. Still not sure about that one.shared class A { void doSomething() synchronized { doSomethingElse(); } private void doSomethingElse() synchronized { } } doSomethingElse must be synchronized even if I as a programmer know it doesn't have to be because the compiler insists it must be. And I know that private methods are visible within the module, but the same rule applies. In essence, we can't avoid recursive mutexes for implementing synchronized, and we're stuck with a lot of recursive locks and unlocks no matter what, as soon as we slap a "shared" label on something.Imagine you have a shared root object that contains a deeply nested private data structure that is technically unshared. Then it becomes not only one more method of the root object that needs to be `synchronized` but it cascades all the way down its private fields as well. One ends up requiring data structures designed for single-threaded execution to grow synchronized methods over night even though they aren't _really_ used concurrently my multiple threads.Oh I see what you're getting at. This sort of thing is why Thread can be initialized with an unshared delegate. Since join() is an implicit synchronization point, it's completely normal to launch a thread that modifies local data, then call join and expect the local data to be in a coherent state. Work queues are much the same.Sure, but at that point they are no longer referenced by the shared Thread, correct?The work items? They stay referenced by the shared Thread until it is done with them. In this particular implementation an item is moved from the list to a separate field that denotes the current item and then the Mutex is released. This current item is technically unshared now, because only this thread can really see it, but as far as the language is concerned there is a shared reference to it because shared applies transitively.The same goes for the list of items while it is under the Mutex protection.Yes. See my suggestion about shared being non-transitive above. I think that's at least in the right ballpark.The rule is simply that you can't be trying to read or write data using both shared and unshared operations, because of that reader-writer contract I mentioned above.Something along that line yes. The exact formulation may need to be ironed out, but what the FAQ says right now doesn't work.Anything between a single shared basic data type and full blown synchronized class is too complicated for the compiler to see through. So a simple definition of `shared` like the FAQ attempts wont fly. Most methods are "somewhat shared": private void workOnAnItem() shared { // m_current is technically never shared, // but we cannot describe this with `shared`. // Hence I manually unshare where appropriate. synchronized (m_condition.unshared.mutex) { m_current = m_list.unshared.front; m_list.unshared.removeFront(); } m_current.unshared.doSomthing(); }As they should be. This is the correct way to use mutexes.You are right, my point was that the original formulation is so strict that can only come from the point of view of using shared in message passing. It doesn't spend a thought on how a shared(Thread) is supposed to be both referable and able to unshare internal lists during processing.And since message passing is encapsulated in an API, we really don't need the type system to do anything special. We can just make correct use an artifact of the API itself.Count me in. Anecdotally I once tried to see if I can write a minimal typed malloc() that is faster than temalloc. It went way over budget from a single CAS instruction, where temalloc mostly works on thread local pools. These synchronizations stall the CPU _that_ much, that I don't see how someone writing lock-free algorithms with `shared` will accept implicit full barriers placed by the language. This is a dead-end to me.Last time I checked boost::shared_ptr (which admittedly was years ago) they'd dropped atomic operations in favor of spins on non-atomics. LOCK has gotten a lot more efficient--I think it's on the order of ~75 cycles these days. And the FENCE operations are supposed to work for normal programming now, though it's hard to find out whether this is true of all CPUs or only those from Intel. But yes, truly atomic ops are terribly expensive.Mostly what I use is load-acquire and store-release, but sometimes raw atomic read access is sufficient as well. So ideally I would like to see: volatile -> compiler doesn't reorder stuffMe too. For example, GCC can optimize around inline assembler. I used to have the inline asm code in core.atomic labeled as volatile for this reason, but was forced to remove it because it's deprecated in D2.and on top of that: atomicLoad/Store -> CPU doesn't reorder stuff in the pipeline in the way I described by MemoryOrder.xxx A shared variable need not be volatile, but a volatile variable is implicitly shared.I'm not quite following you here. Above, I thought you meant the volatile statement. Are you saying we would have both shared and volatile as attributes for variables?
Oct 21 2014
On Tuesday, 21 October 2014 at 16:05:58 UTC, Sean Kelly wrote:Also, of the shared methods that exist, there are synchronized blocks but they occur at a fine grain within the shared methods rather than the entire method being shared. I think that labeling entire methods as synchronized is an inherently flawed concept, as it contradicts the way mutexes are supposed to be used (which is to hold the lock for as short a time as possible).`shared` applies to the implicit `this` parameter, not to the method. It's really no different from normal parameters in this respect, which can either be shared, or not. There's no way to make them temporarily shared either (apart from casting). I think `shared` by itself is fine, as long as it is only take to mean "this method can cope with the parameters being shared". It's `synchronized` and casting that cause the trouble, because they are too coarse, and synchronized doesn't specify which parts of an object it protects. This makes more detailed compiler checks impossible.
Oct 21 2014
"Sean Kelly" <sean invisibleduck.org> On Tuesday, 21 October 2014 at 16:36:30 UTC, Marc Schütz wrote:I think `shared` by itself is fine, as long as it is only take to mean "this method can cope with the parameters being shared". It's `synchronized` and casting that cause the trouble, because they are too coarse, and synchronized doesn't specify which parts of an object it protects. This makes more detailed compiler checks impossible.I think "synchronized" should work basically the same way. For example: class C { int u; shared int s; shared Mutex m; void doSomething() shared { // can be called from a shared reference u = 5; // error, u is not labeled as shared and operation not synchronized s = 5; // ok, though maybe should require s.atomicStore(5) synchronized(m) { u = 5; // ok doSomething2(); // okay because labeled synchronized } } void doSomething2() synchronized { // can be called within a synchronized block u = 5; // ok because synchronized s = 5; // ok, though maybe should require s.atomicStore(5) } } I'd like to avoid having a synchronized label on a method implicitly lock anything because object monitors are terrible and should not exist, and because even if the compiler tries to optimize away recursive locking it won't always happen and the result will be too expensive for anyone to actually want to use it.
Oct 21 2014
Am Tue, 21 Oct 2014 16:05:57 +0000 schrieb "Sean Kelly" <sean invisibleduck.org>:Good point about a shared class not having any unshared methods. =20 I guess that almost entirely eliminates the cases where I might=20 define a class as shared. For example, the MessageBox class in=20 std.concurrency has one or two ostensibly shared methods and the=20 rest are unshared. And it's expected for there to be both shared=20 and unshared references to the object held simultaneously. This=20 is by design, and the implementation would either be horribly=20 slow or straight-up broken if done another way. Also, of the shared methods that exist, there are synchronized=20 blocks but they occur at a fine grain within the shared methods=20 rather than the entire method being shared. I think that=20 labeling entire methods as synchronized is an inherently flawed=20 concept, as it contradicts the way mutexes are supposed to be=20 used (which is to hold the lock for as short a time as possible).=20 I hate to say it, but if I were to apply shared/synchronized=20 labels to class methods it would simply be to service user=20 requests rather than because I think it would actually make the=20 code better or safer.I have nothing to add.=20Yep, and the reason I carefully formulated "read-modify-write", hehe.[=E2=80=A6] I.e. in this case the programmer must decide between mutex synchronization and atomic read-modify-write. That's not too much to ask.=20 I agree. I was being pedantic for the sake of informing anyone=20 who wasn't aware. There are times where I have some fields be=20 lock-free and others protected by a mutex though. See=20 Thread.isRunning, for example.There are times where a write=20 delay is acceptable and the possibility of tearing is irrelevant.=20 But I think this falls pretty squarely into the "expert"=20 category--I don't care if the language makes it easy.=20Do you think it would be bad if a pthread_mutex_t* was declared as shared or only usable when shared ?Imagine you have a shared root object that contains a deeply nested private data structure that is technically unshared. Then it becomes not only one more method of the root object that needs to be `synchronized` but it cascades all the way down its private fields as well. One ends up requiring data structures designed for single-threaded execution to grow synchronized methods over night even though they aren't _really_ used concurrently my multiple threads.=20 I need to give it some more thought, but I think the way this=20 should work is for shared to not be transitive, but for the=20 compiler to require that non-local variables accessed within a=20 shared method must either be declared as shared or the access=20 must occur within a synchronized block. This does trust the=20 programmer a bit more than the current design, but in exchange it=20 encourages a programming model that actually makes sense. It=20 doesn't account for the case where I'm calling pthread_mutex_lock=20 on an unshared variable though. Still not sure about that one.I have to think about that. [=E2=80=A6]The work items? They stay referenced by the shared Thread until it is done with them. In this particular implementation an item is moved from the list to a separate field that denotes the current item and then the Mutex is released. This current item is technically unshared now, because only this thread can really see it, but as far as the language is concerned there is a shared reference to it because shared applies transitively.=20 Oh I see what you're getting at. This sort of thing is why=20 Thread can be initialized with an unshared delegate. Since=20 join() is an implicit synchronization point, it's completely=20 normal to launch a thread that modifies local data, then call=20 join and expect the local data to be in a coherent state. Work=20 queues are much the same.I haven't been around in the D1 times. There was a volatile statement? Anyways what I don't want is that the compiler emits memory barriers everywhere shared variables are accessed. When I use mutex synchronization I don't need it and when I use atomics, I want control over barriers. I thought that could end up in two attributes for variables, but it need not be the case. --=20 MarcoMostly what I use is load-acquire and store-release, but sometimes raw atomic read access is sufficient as well. So ideally I would like to see: volatile -> compiler doesn't reorder stuff=20 Me too. For example, GCC can optimize around inline assembler. =20 I used to have the inline asm code in core.atomic labeled as=20 volatile for this reason, but was forced to remove it because=20 it's deprecated in D2. =20 =20and on top of that: atomicLoad/Store -> CPU doesn't reorder stuff in the pipeline in the way I described by MemoryOrder.xxx A shared variable need not be volatile, but a volatile variable is implicitly shared.=20 I'm not quite following you here. Above, I thought you meant the=20 volatile statement. Are you saying we would have both shared and=20 volatile as attributes for variables?
Oct 21 2014
On Tuesday, 21 October 2014 at 19:32:17 UTC, Marco Leise wrote:Do you think it would be bad if a pthread_mutex_t* was declared as shared or only usable when shared ?No. But the issue in general concerns me. Like say my class contains a C style FILE*. If shared is transitive, then fprintf() must be modified to, what, take shared(FILE*)? Maybe conditionally, since it won't always be shared? I don't know that the shared equivalent of "const" is appropriate here either, since you'd pretty much stick it at the top of every module in core.stdc and core.sys.I haven't been around in the D1 times. There was a volatile statement?http://digitalmars.com/d/1.0/statement.html#VolatileStatementAnyways what I don't want is that the compiler emits memory barriers everywhere shared variables are accessed. When I use mutex synchronization I don't need it and when I use atomics, I want control over barriers.Oh okay. Yes, I'd be fine of all built-in operations were simply prohibited on shared variables, so you were forced to use core.atomic for everything.
Oct 21 2014
Am Tue, 21 Oct 2014 19:43:45 +0000 schrieb "Sean Kelly" <sean invisibleduck.org>:On Tuesday, 21 October 2014 at 19:32:17 UTC, Marco Leise wrote:No no, a FILE* is obviously immutable. -- MarcoDo you think it would be bad if a pthread_mutex_t* was declared as shared or only usable when shared ?No. But the issue in general concerns me. Like say my class contains a C style FILE*. If shared is transitive, then fprintf() must be modified to, what, take shared(FILE*)? Maybe conditionally, since it won't always be shared? I don't know that the shared equivalent of "const" is appropriate here either, since you'd pretty much stick it at the top of every module in core.stdc and core.sys.
Oct 21 2014