digitalmars.D - Implementing multithreading policy templates in D?
- Brian Price (11/11) Jun 07 2008 Greetings,
- Lars Ivar Igesund (8/54) Jun 07 2008 Tango should have most (if not all) the threading primitives you need - ...
- Brian Price (4/38) Jun 07 2008 Thank you for your quick reply. I've looked at Tango and it does seem t...
- Sean Kelly (6/43) Jun 08 2008 when it will be ported to D 2.x?
- downs (4/24) Jun 07 2008 Scrapple.Tools.Threads implements the most important threading primitive...
- Brian Price (9/31) Jun 07 2008 Thanks, it looks good, does it compile & run under D 2.014?
- downs (3/38) Jun 08 2008 Very much agreed.
- Sean Kelly (14/44) Jun 08 2008 porting Loki's Threading Model policies over to D would be just the tick...
- Brian Price (40/89) Jun 08 2008 Problem is that the synchronized keyword does not solve the problem, no ...
- Sean Kelly (28/114) Jun 08 2008 hidden away as they are now) that initialization call would be unnecessa...
- Brian Price (3/123) Jun 08 2008 Would injectMonitor(c, new FakeMutex) be workable if FakeMutex simply im...
Greetings, While implementing a design I ran into a need for multiple implementations with different threading policies. It looked to me as if porting Loki's Threading Model policies over to D would be just the ticket. Unfortunately there's no mutex-like class in Phobos that I can find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized. Without a mutex like object having aquire/release semantics, I'd have to litter my code with a bunch of compile time conditionals instead of using a RAII Lock object. Such a Lock object could be provided via template argument allowing different threading policies for different instantiations. In the 'non-thread-safe' scenario the Lock object would just be an empty object that (I assume) the compiler would optimize away. Since I couldn't figure out how to make such a Lock object using the synchronized statement, I tried rolling my own mutex on top of D 2.014's standard library (Phobos) and language primitives. Since volatile seems to be deprecated, my design uses two synchronization objects through synchronized statements and requires a waitable object that can be signaled by the releasing thread to avoid busy-wait. waitable object I can find in the standard library is Thread and the only event you can wait on is its death. Up to this point the pure D standard mutex implementation was doable (though undoubtedly less efficient than native implementations). Creating a std.Thread derived class to be used as a one shot wait object pushed the design into the yes it will work but it's completely absurd camp. Having used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and need to learn an entirely new approach or there's something missing from the standard library. So I'm left with three questions: Did I miss something in the docs/std lib code? Is there a way to implement flexible threading policies using synchronized statements? What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release? Thanks, Brian
Jun 07 2008
Brian Price wrote:Greetings, While implementing a design I ran into a need for multiple implementations with different threading policies. It looked to me as if porting Loki's Threading Model policies over to D would be just the ticket. Unfortunately there's no mutex-like class in Phobos that I can find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized. Without a mutex like object having aquire/release semantics, I'd have to litter my code with a bunch of compile time conditionals instead of using a RAII Lock object. Such a Lock object could be provided via template argument allowing different threading policies for different instantiations. In the 'non-thread-safe' scenario the Lock object would just be an empty object that (I assume) the compiler would optimize away. Since I couldn't figure out how to make such a Lock object using the synchronized statement, I tried rolling my own mutex on top of D 2.014's standard library (Phobos) and language primitives. Since volatile seems to be deprecated, my design uses two synchronization objects through synchronized statements and requires a waitable object that can be signaled by the releasing thread to avoid busy-wait. only waitable object I can find in the standard library is Thread and the only event you can wait on is its death. Up to this point the pure D standard mutex implementation was doable (though undoubtedly less efficient than native implementations). Creating a std.Thread derived class to be used as a one shot wait object pushed the design into the yes it will work but it's completely absurd camp. Having used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and need to learn an entirely new approach or there's something missing from the standard library. So I'm left with three questions: Did I miss something in the docs/std lib code? Is there a way to implement flexible threading policies using synchronized statements? What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release? Thanks, BrianTango should have most (if not all) the threading primitives you need - to be frank, for any serious threading code, Phobos doesn't cut it. -- Lars Ivar Igesund blog at http://larsivi.net DSource, #d.tango & #D: larsivi Dancing the Tango
Jun 07 2008
Lars Ivar Igesund Wrote:Brian Price wrote:<< -- snip --- >>Greetings, While implementing a design I ran into a need for multiple implementations with different threading policies. It looked to me as if porting Loki's Threading Model policies over to D would be just the ticket. Unfortunately there's no mutex-like class in Phobos that I can find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized.Thank you for your quick reply. I've looked at Tango and it does seem to have an abundance, any idea when it will be ported to D 2.x? BrianHaving used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and need to learn an entirely new approach or there's something missing from the standard library. So I'm left with three questions: Did I miss something in the docs/std lib code? Is there a way to implement flexible threading policies using synchronized statements? What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release? Thanks, BrianTango should have most (if not all) the threading primitives you need - to be frank, for any serious threading code, Phobos doesn't cut it. -- Lars Ivar Igesund blog at http://larsivi.net DSource, #d.tango & #D: larsivi Dancing the Tango
Jun 07 2008
== Quote from Brian Price (blprice61 yahoo.com)'s articleLars Ivar Igesund Wrote:when it will be ported to D 2.x? "Someday." Doing so will take time, and I don't even have enough of that to fix the bugs already on my plate. That and, to be frank, supporting D 2.0 isn't a terribly high priority. Perhaps if GDC had better 2.0 support. SeanBrian Price wrote:<< -- snip --- >>Greetings, While implementing a design I ran into a need for multiple implementations with different threading policies. It looked to me as if porting Loki's Threading Model policies over to D would be just the ticket. Unfortunately there's no mutex-like class in Phobos that I can find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized.Thank you for your quick reply. I've looked at Tango and it does seem to have an abundance, any ideaHaving used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and need to learn an entirely new approach or there's something missing from the standard library. So I'm left with three questions: Did I miss something in the docs/std lib code? Is there a way to implement flexible threading policies using synchronized statements? What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release? Thanks, BrianTango should have most (if not all) the threading primitives you need - to be frank, for any serious threading code, Phobos doesn't cut it. -- Lars Ivar Igesund blog at http://larsivi.net DSource, #d.tango & #D: larsivi Dancing the Tango
Jun 08 2008
Brian Price wrote:Greetings, While implementing a design I ran into a need for multiple implementations with different threading policies. It looked to me as if porting Loki's Threading Model policies over to D would be just the ticket. Unfortunately there's no mutex-like class in Phobos that I can find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized. Without a mutex like object having aquire/release semantics, I'd have to litter my code with a bunch of compile time conditionals instead of using a RAII Lock object. Such a Lock object could be provided via template argument allowing different threading policies for different instantiations. In the 'non-thread-safe' scenario the Lock object would just be an empty object that (I assume) the compiler would optimize away. Since I couldn't figure out how to make such a Lock object using the synchronized statement, I tried rolling my own mutex on top of D 2.014's standard library (Phobos) and language primitives. Since volatile seems to be deprecated, my design uses two synchronization objects through synchronized statements and requires a waitable object that can be signaled by the releasing thread to avoid busy-wait. waitable object I can find in the standard library is Thread and the only event you can wait on is its death. Up to this point the pure D standard mutex implementation was doable (though undoubtedly less efficient than native implementations). Creating a std.Thread derived class to be used as a one shot wait object pushed the design into the yes it will work but it's completely absurd camp. Having used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and need to learn an entirely new approach or there's something missing from the standard library. So I'm left with three questions: Did I miss something in the docs/std lib code? Is there a way to implement flexible threading policies using synchronized statements? What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release? Thanks, BrianScrapple.Tools.Threads implements the most important threading primitives on Win32 and Posix. http://dsource.org/projects/scrapple/browser/trunk/tools/tools/threads.d --downs
Jun 07 2008
downs Wrote:Brian Price wrote:<<---- snip ---- >>Greetings, While implementing a design I ran into a need for multiple implementations with different threading policies. It looked to me as if porting Loki's Threading Model policies over to D would be just the ticket. Unfortunately there's no mutex-like class in Phobos that I can find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized.Thanks, it looks good, does it compile & run under D 2.014? For the moment I'm using the simple hack: extern (C) void _d_monitorenter(Object obj); extern (C) void _d_monitorexit(Object obj); for a basic acquire/release mutex, but as my needs expand I want to avoid reinventing the wheel. I'd be a lot more comfortable though if the powers that be would expose those two functions as methods on Object. BrianHaving used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and need to learn an entirely new approach or there's something missing from the standard library. So I'm left with three questions: Did I miss something in the docs/std lib code? Is there a way to implement flexible threading policies using synchronized statements? What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release? Thanks, BrianScrapple.Tools.Threads implements the most important threading primitives on Win32 and Posix. http://dsource.org/projects/scrapple/browser/trunk/tools/tools/threads.d --downs
Jun 07 2008
Brian Price wrote:downs Wrote:No idea, I never tried - GDC 2.0 support is spotty.Brian Price wrote:<<---- snip ---- >>Greetings, While implementing a design I ran into a need for multiple implementations with different threading policies. It looked to me as if porting Loki's Threading Model policies over to D would be just the ticket. Unfortunately there's no mutex-like class in Phobos that I can find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized.Thanks, it looks good, does it compile & run under D 2.014?Having used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and need to learn an entirely new approach or there's something missing from the standard library. So I'm left with three questions: Did I miss something in the docs/std lib code? Is there a way to implement flexible threading policies using synchronized statements? What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release? Thanks, BrianScrapple.Tools.Threads implements the most important threading primitives on Win32 and Posix. http://dsource.org/projects/scrapple/browser/trunk/tools/tools/threads.d --downsFor the moment I'm using the simple hack: extern (C) void _d_monitorenter(Object obj); extern (C) void _d_monitorexit(Object obj); for a basic acquire/release mutex, but as my needs expand I want to avoid reinventing the wheel. I'd be a lot more comfortable though if the powers that be would expose those two functions as methods on Object.Very much agreed.Brian
Jun 08 2008
== Quote from Brian Price (blprice61 yahoo.com)'s articledowns Wrote:porting Loki's Threading Model policies over to D would be just the ticket. Unfortunately there's no mutex-like class in Phobos that I can find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized.Brian Price wrote:Greetings, While implementing a design I ran into a need for multiple implementations with different threading policies. It looked to me as ifto learn an entirely new approach or there's something missing from the standard library. So I'm left with three questions:<<---- snip ---- >>Having used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and needI gave this a lot of thought, and decided that building the functionality into Object is a bad idea. However, the Mutex classes in Tango are integrated with the "synchronized" statement, which produces a similar result: auto m = new Mutex; auto c = new Condition( m ); synchronized( m ) { c.wait; } SeanThanks, it looks good, does it compile & run under D 2.014? For the moment I'm using the simple hack: extern (C) void _d_monitorenter(Object obj); extern (C) void _d_monitorexit(Object obj); for a basic acquire/release mutex, but as my needs expand I want to avoid reinventing the wheel. I'd be a lot more comfortable though if the powers that be would expose those two functions as methods on Object.Did I miss something in the docs/std lib code? Is there a way to implement flexible threading policies using synchronized statements? What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release? Thanks, BrianScrapple.Tools.Threads implements the most important threading primitives on Win32 and Posix. http://dsource.org/projects/scrapple/browser/trunk/tools/tools/threads.d --downs
Jun 08 2008
Sean Kelly Wrote:== Quote from Brian Price (blprice61 yahoo.com)'s articleProblem is that the synchronized keyword does not solve the problem, no matter how it's used. Why should different template class specializations be forced to pay the synchronization penalty when some may be used in an inherently thread safe manner, while others may need class level locking, and still others object level locking? Usage example follows: template ObjectLockablePolicy( MutexType = DefaultMutexPolicy ) { private MutexType mutex_; private void initSynchPolicy() { mutex_ = new MutexType(); } scope class Lock { this() { mutex_.acquire(); } ~this() { mutex_.release(); } } } class SomeClass { int a,b; mixin ClassLockablePolicy; this() { initSynchPolicy(); } void somefunc() { scope Lock lock = new Lock(); writefln("inside scoped lock"); } } This creates the equivalent of a synchronized(mutex_) block around writefln, but consider the ugliness necessary with initSynchPolicy(). If the methods were exposed on Object (instead of hidden away as they are now) that initialization call would be unnecessary. Lock's constructor/destructor could simply call acquire/release on the outer this pointer in this particular case. It'd be different if every Object wasn't built on top of a monitor anyhow, but not only is it true of the current implementation, but must be true for any implementation as long as sychronized(someObjectInstance) is allowed. Sincerely, Brian Pricedowns Wrote:porting Loki's Threading Model policies over to D would be just the ticket. Unfortunately there's no mutex-like class in Phobos that I can find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized.Brian Price wrote:Greetings, While implementing a design I ran into a need for multiple implementations with different threading policies. It looked to me as ifto learn an entirely new approach or there's something missing from the standard library. So I'm left with three questions:<<---- snip ---- >>Having used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and needI gave this a lot of thought, and decided that building the functionality into Object is a bad idea. However, the Mutex classes in Tango are integrated with the "synchronized" statement, which produces a similar result: auto m = new Mutex; auto c = new Condition( m ); synchronized( m ) { c.wait; } SeanThanks, it looks good, does it compile & run under D 2.014? For the moment I'm using the simple hack: extern (C) void _d_monitorenter(Object obj); extern (C) void _d_monitorexit(Object obj); for a basic acquire/release mutex, but as my needs expand I want to avoid reinventing the wheel. I'd be a lot more comfortable though if the powers that be would expose those two functions as methods on Object.Did I miss something in the docs/std lib code? Is there a way to implement flexible threading policies using synchronized statements? What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release? Thanks, BrianScrapple.Tools.Threads implements the most important threading primitives on Win32 and Posix. http://dsource.org/projects/scrapple/browser/trunk/tools/tools/threads.d --downs
Jun 08 2008
== Quote from Brian Price (blprice61 yahoo.com)'s articleSean Kelly Wrote:penalty when some may be used in an inherently thread safe manner, while others may need class level locking, and still others object level locking? Usage example follows:== Quote from Brian Price (blprice61 yahoo.com)'s articleProblem is that the synchronized keyword does not solve the problem, no matter how it's used. Why should different template class specializations be forced to pay the synchronizationdowns Wrote:porting Loki's Threading Model policies over to D would be just the ticket. Unfortunately there's no mutex-like class in Phobos that I can find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized.Brian Price wrote:Greetings, While implementing a design I ran into a need for multiple implementations with different threading policies. It looked to me as ifto learn an entirely new approach or there's something missing from the standard library. So I'm left with three questions:<<---- snip ---- >>Having used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and needI gave this a lot of thought, and decided that building the functionality into Object is a bad idea. However, the Mutex classes in Tango are integrated with the "synchronized" statement, which produces a similar result: auto m = new Mutex; auto c = new Condition( m ); synchronized( m ) { c.wait; } SeanThanks, it looks good, does it compile & run under D 2.014? For the moment I'm using the simple hack: extern (C) void _d_monitorenter(Object obj); extern (C) void _d_monitorexit(Object obj); for a basic acquire/release mutex, but as my needs expand I want to avoid reinventing the wheel. I'd be a lot more comfortable though if the powers that be would expose those two functions as methods on Object.Did I miss something in the docs/std lib code? Is there a way to implement flexible threading policies using synchronized statements? What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release? Thanks, BrianScrapple.Tools.Threads implements the most important threading primitives on Win32 and Posix. http://dsource.org/projects/scrapple/browser/trunk/tools/tools/threads.d --downstemplate ObjectLockablePolicy( MutexType = DefaultMutexPolicy ) { private MutexType mutex_; private void initSynchPolicy() { mutex_ = new MutexType(); } scope class Lock { this() { mutex_.acquire(); } ~this() { mutex_.release(); } } } class SomeClass { int a,b; mixin ClassLockablePolicy; this() { initSynchPolicy(); } void somefunc() { scope Lock lock = new Lock(); writefln("inside scoped lock"); } } This creates the equivalent of a synchronized(mutex_) block around writefln, but consider the ugliness necessary with initSynchPolicy(). If the methods were exposed on Object (instead ofhidden away as they are now) that initialization call would be unnecessary. Lock's constructor/destructor could simply call acquire/release on the outer this pointer in this particular case.It'd be different if every Object wasn't built on top of a monitor anyhow, but not only is it true of the current implementation, but must be true for any implementation as long assychronized(someObjectInstance) is allowed. One interesting aspect of the Tango approach is that you can provide the monitor object for any object in D. The motivation for this was similar to what you're saying above. By building the functionality into Object directly (and therefore into the runtime), the user is constrained by the assumptions made by the runtime implementation. By contrast, the Tango approach allows the user to provide any monitor implementation they wish, as need dictates. My original motivation for this was to allow objects to be allocated in shared memory and have "synchronized" still work with them, but the same approach allows for the use of futexes, etc. Interestingly, the Tango Mutex is actually also its own monitor because of this approach. However, in general I was thinking along these lines: class C { void foo() { synchronized { // A } } } auto c = new C; injectMonitor( c, new Mutex ); c.foo(); The injectMonitor call is just a one-line function. So at point A, the user-provided mutex is locked. In fact, the default runtime- provided monitor is never constructed at all. I grant that requiring injectMonitor to be called may be a bit more complicated than some other approaches, but I feel it's also more flexible. Sean
Jun 08 2008
Sean Kelly Wrote:== Quote from Brian Price (blprice61 yahoo.com)'s articleWould injectMonitor(c, new FakeMutex) be workable if FakeMutex simply implemented the required interface without providing any functionality? That would, I think, allow the compiler to optimize away the entire synchronized construct for the 'thread safe usage' case. Still, its use requires one statement per instance at point of creation whereas the mixin approach only requires one statement per template class. Perhaps it would be possible to call injectMonitor from within the constructor? BrianSean Kelly Wrote:penalty when some may be used in an inherently thread safe manner, while others may need class level locking, and still others object level locking? Usage example follows:== Quote from Brian Price (blprice61 yahoo.com)'s articleProblem is that the synchronized keyword does not solve the problem, no matter how it's used. Why should different template class specializations be forced to pay the synchronizationdowns Wrote:porting Loki's Threading Model policies over to D would be just the ticket. Unfortunately there's no mutex-like class in Phobos that I can find and Object's monitor is not exposed so no way to acquire/release on it other than through synchronized.Brian Price wrote:Greetings, While implementing a design I ran into a need for multiple implementations with different threading policies. It looked to me as ifto learn an entirely new approach or there's something missing from the standard library. So I'm left with three questions:<<---- snip ---- >>Having used about every 'mainstream' language over the past twenty odd years, I figure either I'm missing something huge and needI gave this a lot of thought, and decided that building the functionality into Object is a bad idea. However, the Mutex classes in Tango are integrated with the "synchronized" statement, which produces a similar result: auto m = new Mutex; auto c = new Condition( m ); synchronized( m ) { c.wait; } SeanThanks, it looks good, does it compile & run under D 2.014? For the moment I'm using the simple hack: extern (C) void _d_monitorenter(Object obj); extern (C) void _d_monitorexit(Object obj); for a basic acquire/release mutex, but as my needs expand I want to avoid reinventing the wheel. I'd be a lot more comfortable though if the powers that be would expose those two functions as methods on Object.Did I miss something in the docs/std lib code? Is there a way to implement flexible threading policies using synchronized statements? What are the chances we'll see Object sporting wait/notify methods or lock/unlock methods in a future release? Thanks, BrianScrapple.Tools.Threads implements the most important threading primitives on Win32 and Posix. http://dsource.org/projects/scrapple/browser/trunk/tools/tools/threads.d --downstemplate ObjectLockablePolicy( MutexType = DefaultMutexPolicy ) { private MutexType mutex_; private void initSynchPolicy() { mutex_ = new MutexType(); } scope class Lock { this() { mutex_.acquire(); } ~this() { mutex_.release(); } } } class SomeClass { int a,b; mixin ClassLockablePolicy; this() { initSynchPolicy(); } void somefunc() { scope Lock lock = new Lock(); writefln("inside scoped lock"); } } This creates the equivalent of a synchronized(mutex_) block around writefln, but consider the ugliness necessary with initSynchPolicy(). If the methods were exposed on Object (instead ofhidden away as they are now) that initialization call would be unnecessary. Lock's constructor/destructor could simply call acquire/release on the outer this pointer in this particular case.It'd be different if every Object wasn't built on top of a monitor anyhow, but not only is it true of the current implementation, but must be true for any implementation as long assychronized(someObjectInstance) is allowed. One interesting aspect of the Tango approach is that you can provide the monitor object for any object in D. The motivation for this was similar to what you're saying above. By building the functionality into Object directly (and therefore into the runtime), the user is constrained by the assumptions made by the runtime implementation. By contrast, the Tango approach allows the user to provide any monitor implementation they wish, as need dictates. My original motivation for this was to allow objects to be allocated in shared memory and have "synchronized" still work with them, but the same approach allows for the use of futexes, etc. Interestingly, the Tango Mutex is actually also its own monitor because of this approach. However, in general I was thinking along these lines: class C { void foo() { synchronized { // A } } } auto c = new C; injectMonitor( c, new Mutex ); c.foo(); The injectMonitor call is just a one-line function. So at point A, the user-provided mutex is locked. In fact, the default runtime- provided monitor is never constructed at all. I grant that requiring injectMonitor to be called may be a bit more complicated than some other approaches, but I feel it's also more flexible. Sean
Jun 08 2008