www.digitalmars.com         C & C++   DMDScript  

digitalmars.D.learn - Parallel thread safety (fun fun)

reply "Mineko" <uminekorox gmail.com> writes:
So, I'm implementing some parallelism in my engine (maybe some 
concurrency where appropriate later), and I'm having some issues 
with thread safety, and synchronize ain't cutting it.

What I figure is that if I can get the IO class working within a 
parallel thread then everything else should fall in place, so, I 
need some help making the IO class thread-safe. (I'm new to 
thread safety btw, like, yesterday new >_>)

Relevent file here: 
https://github.com/MinekoRox/Breaker-Engine/blob/master/src/breaker/utility/io.d

Feel free to drop any other suggestions on what to do with that 
class if you want, always appreciated. :)
Jan 31 2014
parent reply "TheFlyingFiddle" <theflyingfiddle gmail.com> writes:
On Friday, 31 January 2014 at 21:33:50 UTC, Mineko wrote:

 So, I'm implementing some parallelism in my engine (maybe some 
 concurrency where appropriate later), and I'm having some 
 issues with thread safety, and synchronize ain't cutting it.

 What I figure is that if I can get the IO class working within 
 a parallel thread then everything else should fall in place, 
 so, I need some help making the IO class thread-safe. (I'm new 
 to thread safety btw, like, yesterday new >_>)

 Relevent file here: 
 https://github.com/MinekoRox/Breaker-Engine/blob/master/src/breaker/utility/io.d

 Feel free to drop any other suggestions on what to do with that 
 class if you want, always appreciated. :)


First of all you are using the synchronized statement incorrecly.

synchronized bool addTime; <-- This does absolutely nothing.

I am supprised this code is allowed.

You can only use synchronized on class methods 
http://dlang.org/class.html#synchronized-functions or on an 
abitrary
block of code 
http://dlang.org/statement.html#SynchronizedStatement

If you want to synchronize access to data you will have to either 
make them shared and use  synchronized/atomic operations on them 
or synchronize the use yourself. This is a quite complex and 
bugprone activity and if your just getting started i would not 
reccomend it. There is a simpler / better way that i will 
describe later.

Secondly the only use of concurency that i see in the IO class is 
the attempt to write to a file from multiple threads. In the 
write() method. This is not a good idea. Firstly you will not be 
able to write any faster to the file since it's IO bound in any 
case. The only effect if any is that you will use more resources 
and that the output will appear in the file in random order. 
(Probably not what you want)


I would reccomend using message passing instead of even 
attempting to synchronize things. This is ALOT simpler and if 
your goal is to offload IO to another thread it's the way to go 
IMHO.

Here is a very simple logging application that writes to a file 
using std.concurrency through message passing. To make things 
simple it can only log messages from the thread that started it. 
This would probably be the main thread in a game.

module main;
import logger;

void main()
{
     startLoggerThread("log.txt");

     log("Hello");
     log("World");

     stopLoggerThread();
}

//logger.d
module logger;

import std.concurrency;
import std.file;

//This is a unique identifier for a thread.
Tid loggerTid;

//Starts the logger thread.
//(This must be done before any logging can take place.)
void startLoggerThread(string logFile)
{
     loggerTid = spawn(&loggerEntry, logFile);
}

//Stops the logger thread.
void stopLoggerThread()
{
     send(loggerTid, Shutdown());
}

//Called by the thread that started the logger.
void log(string msg)
{
     //Send the message to log to the logger thread
     //identified by the loggerTid.
     send(loggerTid, msg);
}

//A tag struct symbolising the message shutdown.
struct Shutdown { }

void loggerEntry(string logFile)
{
     bool running = true;
     //Recevie logging requests until stopLoggerThread is called.
     while(running)
     {
     	receive(
     	   //We get a message here when log is called.
     	   (string msg)
     	   {
                //Create file if it does not already exists
     	       //otherwise append to it.
     	       if(!exists(logFile))
     	           write(logFile, msg ~ "\n");
     	       else
     		   append(logFile, msg ~ "\n");
     	    },
     	    //If we get this message we stop the logger thread.
     	    (Shutdown _)
     	    {
                running = false;
     	    });
     }
}

The example works by first creating a logging thread. This thread 
then listens to logging requests from the main thread. And logs 
all messages it receives.

I would reccomend reading the TDPL chapter on concurrency if you 
want to learn more.

//Hope this was all helpful
Feb 01 2014
parent "Mineko" <uminekorox gmail.com> writes:
On Saturday, 1 February 2014 at 19:26:03 UTC, TheFlyingFiddle 
wrote:

 On Friday, 31 January 2014 at 21:33:50 UTC, Mineko wrote:

 So, I'm implementing some parallelism in my engine (maybe some 
 concurrency where appropriate later), and I'm having some 
 issues with thread safety, and synchronize ain't cutting it.

 What I figure is that if I can get the IO class working within 
 a parallel thread then everything else should fall in place, 
 so, I need some help making the IO class thread-safe. (I'm new 
 to thread safety btw, like, yesterday new >_>)

 Relevent file here: 
 https://github.com/MinekoRox/Breaker-Engine/blob/master/src/breaker/utility/io.d

 Feel free to drop any other suggestions on what to do with 
 that class if you want, always appreciated. :)


 First of all you are using the synchronized statement 
 incorrecly.

 synchronized bool addTime; <-- This does absolutely nothing.

 I am supprised this code is allowed.

 You can only use synchronized on class methods 
 http://dlang.org/class.html#synchronized-functions or on an 
 abitrary
 block of code 
 http://dlang.org/statement.html#SynchronizedStatement

 If you want to synchronize access to data you will have to 
 either make them shared and use  synchronized/atomic operations 
 on them or synchronize the use yourself. This is a quite 
 complex and bugprone activity and if your just getting started 
 i would not reccomend it. There is a simpler / better way that 
 i will describe later.

 Secondly the only use of concurency that i see in the IO class 
 is the attempt to write to a file from multiple threads. In the 
 write() method. This is not a good idea. Firstly you will not 
 be able to write any faster to the file since it's IO bound in 
 any case. The only effect if any is that you will use more 
 resources and that the output will appear in the file in random 
 order. (Probably not what you want)


 I would reccomend using message passing instead of even 
 attempting to synchronize things. This is ALOT simpler and if 
 your goal is to offload IO to another thread it's the way to go 
 IMHO.

 Here is a very simple logging application that writes to a file 
 using std.concurrency through message passing. To make things 
 simple it can only log messages from the thread that started 
 it. This would probably be the main thread in a game.

 module main;
 import logger;

 void main()
 {
     startLoggerThread("log.txt");

     log("Hello");
     log("World");

     stopLoggerThread();
 }

 //logger.d
 module logger;

 import std.concurrency;
 import std.file;

 //This is a unique identifier for a thread.
 Tid loggerTid;

 //Starts the logger thread.
 //(This must be done before any logging can take place.)
 void startLoggerThread(string logFile)
 {
     loggerTid = spawn(&loggerEntry, logFile);
 }

 //Stops the logger thread.
 void stopLoggerThread()
 {
     send(loggerTid, Shutdown());
 }

 //Called by the thread that started the logger.
 void log(string msg)
 {
     //Send the message to log to the logger thread
     //identified by the loggerTid.
     send(loggerTid, msg);
 }

 //A tag struct symbolising the message shutdown.
 struct Shutdown { }

 void loggerEntry(string logFile)
 {
     bool running = true;
     //Recevie logging requests until stopLoggerThread is called.
     while(running)
     {
     	receive(
     	   //We get a message here when log is called.
     	   (string msg)
     	   {
                //Create file if it does not already exists
     	       //otherwise append to it.
     	       if(!exists(logFile))
     	           write(logFile, msg ~ "\n");
     	       else
     		   append(logFile, msg ~ "\n");
     	    },
     	    //If we get this message we stop the logger thread.
     	    (Shutdown _)
     	    {
                running = false;
     	    });
     }
 }

 The example works by first creating a logging thread. This 
 thread then listens to logging requests from the main thread. 
 And logs all messages it receives.

 I would reccomend reading the TDPL chapter on concurrency if 
 you want to learn more.

 //Hope this was all helpful


Yes it was, this was more or less what I was looking for, 
especially about the synchronization part, thank you.
Feb 01 2014